Kove research on kdb+ and file systems
Kove uses STAC-M3 with their DRAM-based storage to study the effect of volume and directory configuration on kdb+ performance.
Solid-state drives are an increasingly interesting component for tick database solutions, but finding optimal software configurations to exploit them is fairly virgin territory. Using their XPD2 DRAM-based storage, Kove decided to explore what effect varying the number of volumes and directories might have on the query response times of kdb+. They hypothesized two effects:
1. The Linux kernel block device layer performs I/O scheduling separately for each block device. Varying the number of storage devices (and thus block devices) attached to the system varies the level of storage device I/O parallelism in the Linux kernel.
2. One configures kdb+ to use a set of directories for data storage, and each kdb+ server instance assigns a subset of those directories to each of its threads. Thus varying the number of directories varies the level of I/O parallelism in kdb+.
The baseline STAC-M3 suite (Antuco) contains a range of benchmarks with varying patterns of serial and random I/O, as well as compute intensity. Kove tested each of these benchmarks against several different volume/directory configurations. The methodology, system configuration, and results are presented in a 19-page report containing several tables and charts.
Premium STAC member firms may request the report from the STAC Vault. It is also available to Observer Member firms that have not already obtained their complimentary report from the STAC Vault.
To inquire about your firm's membership status or to request an upgrade, please contact us.