Midtown Blogger/Manhattan Valley Follies: Extreme Tech- Problem of "Garbage Collection"

EXTREMETECH
COMPUTING
RESEARCHERS: SSDS STRUGGLE IN VIRTUAL MACHINES THANKS TO GARBAGE COLLECTION

Researchers: SSDs struggle in virtual machines thanks to garbage collection

By Joel Hruska on May 27, 2015 at 3:42 pm
Comment

Share This Article

The advent of solid state drives has revolutionized the storage business at virtually every level. Whether you’re running a modest laptop or a corporate server, using an SSD instead of a conventional spinning disk can improve power consumption and vastly boost performance. One area where SSDs have a particular advantage over their magnetic brethren is in serving multiple simultaneous I/O requests or hosting several virtual machines simultaneously — but new research shows that consumer SSDs struggle to fulfill these tasks over the long term thanks to garbage collection algorithms.

First, let’s talk about the overarching problem. It’s easy to see why running multiple VMs on a single conventional hard drive quickly leads to performance degradation — each VM is competing for access to a single spinning disk. RAID arrays can mitigate this problem to an extent, but using multiple disks introduces its own variables. SSDs, with their high speed interfaces and ability to access data hundreds of times faster than a spinning disk, should be a perfect fit for this kind of workload. What researchers found, however, is that the garbage collection routines that run on modern SSDs actually make them a poor fit for these workloads.

There’s actually two problems at work here. First, as with most workloads, as an SSD ages, performance degrades. Figure 2 below is drawn from the initial report, it examines performance in three Virtual Machine instances (each focused on a different type of workload). Running all three together at the same time (concurrent) reduces performance in both cases, but the performance hit from running them in this fashion after aging the drives by performing a series of random writes (at 4K-32K sizes) drops dramatically.

This is where garbage collection and other algorithms are specifically designed to restore performance, but the researchers report that garbage collection runs into problems precisely because of the disconnect between what the SSD sees as idle blocks ripe for picking and the needs of the host VM running on top of the host OS, which is ultimately running on the SSD. Because concurrent VM session data can mixed internally — which is to say, information from multiple VMs can sit side by side within the same block of NAND flash — garbage collection can impact all of the VM sessions simultaneously. The graph below illustrates how data was laid out within the SSD structure and the number of pages that were touched by the ongoing garbage collection (thereby impacting drive performance):

The paper authors conclude that guaranteeing a set number of IOPS when multiple VMs are hosted on a single consumer SSD is effectively impossible. While they don’t test multiple SSDs or some of the high-end commercial offerings available from Intel or other vendors, nothing about the architecture of those drives suggests they would be unaffected. The absolute level of performance might be much higher, but the same tendency to mix I/O from different VM sessions at the drive level is going to plague the results.

Better SSD controllers will drive performance in the future

One of the most profound differences between SSDs and HDDs has little to do with the storage medium itself. Hard drive controllers aren’t simple, exactly, but they don’t track as many variables as SSD controllers do, either. A modern SSD controller manages 8-16 parallel channels to NAND and makes decisions about garbage collection, wear leveling, and error recovery. Maximizing performance of the internal interface means juggling a number of variables in real time.

A typical NAND flash controller

We’ve talked before about the long-term barriers to NAND flash scaling. Unlike microprocessors, which have tended to increase performance at lower process nodes, shrinking NAND flash to smaller geometries doesn’t make it faster — if anything, it tends to make it slightly slower. Fewer electrons trapped per cell means that some of the compensatory mechanisms slow things down a bit, as does increased error checking.

The performance increases we’ve seen from SSDs, leaving out achievements like Samsung’s 3D NAND, have come from improved SSD controllers, the use of SLC blocks to speed data transfers, and increased drive bandwidth. What the researchers in this paper recommend for VMs is a controller implementation in which specific, dedicated blocks of NAND flash within the same physical drive are set aside for each individual VM. In theory, a future drive controller could offer that kind of fine-grained capability, though it would likely be reserved for commercial hardware for obvious reasons.

This kind of improvement, while enterprise-focused, is where performance accelerations in NAND storage will likely come from in years ahead. Boosting drive bandwidth works to a point, but past that, better controllers will have to deliver better performance — at least until new memory technologies come online.