Abstract
SQL databases have long dominated the database landscape, but the recent rise of NOSQL databases, particularly key-value stores, has left benchmarking technology playing catch-up. Despite the popularity of key-value stores, existing workload generators and benchmarks fail to model the characteristics of real production workloads. For instance, YCSB, KVbench, and db_bench cannot generate dynamic workloads with requirements that shift over time, workloads with composite keys and values with different prefix distributions, and data with varying degrees of sortedness. On top of these major limitations, each workload generator/benchmark has its own specific issues: YCSB fails to support all the types of possible operations on KV stores, KVbench lacks support for running its operations on a specific database, and db_bench is tightly coupled with RocksDB. To address these limitations, I designed Tectonic+: A benchmark for key-value stores with the ability to generate more realistic, dynamic workloads that can model changes to data access patterns over time. Tectonic+ provides extensive customization for all aspects of workload generation and supports a broad range of key-value operations across multiple databases out of the box. Tectonic+ achieves this while offering up to 2x more throughput and using 88% less memory than the state of the art. By allowing for the simulation of more realistic workloads, Tectonic+ enables users to quickly and accurately assess how their production workload will perform against different databases with varied configurations. I benchmarked Tectonic+’s performance against YCSB by measuring and comparing throughput, peak memory usage, and CPU utilization. The code for Tectonic+ is available at https://github.com/SSD-Brandeis/Tectonic.