Today's key-value stores are either disk-optimized, focusing on large data and saturating device IOPS, or memory-optimized, focusing on high throughput with linear thread scaling assuming plenty of main memory. However, many practical workloads demand high performance for read and write working sets that are much larger than main memory, over a total data size that is even larger. They require judicious use of memory and disk, and today's systems do not handle such workloads well. We present F2, a new key-value store design based on compartmentalization -- it consists of five key components that work together in well-defined ways to achieve high throughput -- saturating disk and memory bandwidths -- while incurring low disk read and write amplification. A key design characteristic of F2 is that it separates the management of hot and cold data, across the read and write domains, and adapts the use of memory to optimize each case. Through a sequence of new latch-free system constructs, F2 solves the key challenge of maintaining high throughput with linear thread scalability in such a compartmentalized design. Detailed experiments on benchmark data validate our design's superiority, in terms of throughput, over state-of-the-art key-value stores, when the available memory resources are scarce.
翻译:今天的键-值存储器要么是以磁盘为基础,专注于大数据和饱和设备 IOPS,要么是以内存为基础,专注于高吞吐量,线性线程扩展,假定有大量的主内存。然而,许多实际工作负载需要对比主内存大得多的读写工作集进行高性能处理,其总数据量更大。它们需要谨慎地使用内存和磁盘,而今天的系统无法很好地处理这些工作负载。我们提出了 F2,一种基于隔离的新的键-值存储器设计——它由五个关键组件组成,相互协作,以明确定义的方式实现高吞吐量——饱和磁盘和内存带宽,同时产生低磁盘读写放大。F2 的一个关键设计特性是它将热数据和冷数据的管理分离到读和写域中,并根据需要调整内存使用、优化热数据和冷数据的读写性能。通过一系列新的无锁系统构造,F2 解决了在这种隔离设计中保持高吞吐量的关键挑战。基准数据上的详细实验验证了我们的设计在内存资源有限时的吞吐量优越性,相比我们公认的键-值存储器。