Kove research on kdb+ and file systems

Kove uses STAC-M3 with their DRAM-based storage to study the effect of volume and directory configuration on kdb+ performance.

Solid-state drives are an increasingly interesting component for tick database solutions, but finding optimal software configurations to exploit them is fairly virgin territory. Using their XPD2 DRAM-based storage, Kove decided to explore what effect varying the number of volumes and directories might have on the query response times of kdb+. They hypothesized two effects:

1. The Linux kernel block device layer performs I/O scheduling separately for each block device. Varying the number of storage devices (and thus block devices) attached to the system varies the level of storage device I/O parallelism in the Linux kernel.

2. One configures kdb+ to use a set of directories for data storage, and each kdb+ server instance assigns a subset of those directories to each of its threads. Thus varying the number of directories varies the level of I/O parallelism in kdb+.

The baseline STAC-M3 suite (Antuco) contains a range of benchmarks with varying patterns of serial and random I/O, as well as compute intensity. Kove tested each of these benchmarks against several different volume/directory configurations. The methodology, system configuration, and results are presented in a 19-page report containing several tables and charts.

Qualified STAC subscribers may access the report here. 

To inquire about your firm's membership status or to request an upgrade, please contact us