Each year, the capital markets get more algorithmic. Whether it's trading, investing, or managing risk and collateral, the pressures of competition and regulation are fueling a need for increasingly massive computations and increasingly huge amounts of data.
For this reason, large firms (and even some smaller ones) have datacenters packed with tens or hundreds of thousands of cores dedicated to quantitative analysis, along with petabytes of storage, while many of them are starting to leverage cloud technologies.
A firm's goals with respect to quant infrastructure can depend on the extent to which it is focused on revenue expansion, cost reduction, or regulatory compliance. It may wish to analyze more information, increase the accuracy of calculations, reduce the cost per calculation, or all of the above. No matter what the requirement, technology innovation has proven to be a crucial enabler.
Quants want to get as much scale and performance as they can from languages such as q and Python and software frameworks such as Apache Spark. And they are increasingly incorporating artificial intelligence tools into the mix as well.
On the compute side, new processors, memory, and interconnects, as well as innovative libraries, development tools, and middleware can create favorable shifts in tradeoff curves (e.g., accuracy vs speed, throughput vs power consumption, etc.). The STAC-A2 benchmark suite, based on market risk analytics, has become the standard for measuring the performance, price-performance, and resource efficiency of diverse architectures when handling compute-intensive workloads.
On the data side, new storage media and high-speed interconnects, along with filesystems that can leverage them, are revolutionizing what can be done with big data. Because so much of quant finance is about predicting the future from the past, time series data is the lifeblood of the industry. STAC has two benchmarks based on time series analysis. STAC-M3 is a collection of enterprise tick analytics inspired by use cases ranging from strategy development and TCA to execution quality analysis and risk management. STAC-A3 is a set of benchmarks based on backtesting workloads (developing trading strategies and testing them on out-of-sample data).