NVMe NAS Cache: Higher Speed or More Capacity?

Introduction, Specifications, and Pricing

More consumer and prosumer-grade network attached storage (NAS) ship with the ability to accelerate workloads with SSDs. We can't pinpoint when the feature rolled out because the software feature was rolled out on existing products and not artificially limited to new models. Chances are, if you own a NAS manufactured in the last four years, the software supports using an SSD cache.

38

The latest cache acceleration feature from NAS manufactures is to expand the SSD options to include high-speed NVMe drives. Our QNAP TS-EC1279U-RP first reviewed here in May 2012 added this feature even though the system came to market long before the first NVMe SSD shipped. With NVMe and 40GbE network capabilities, we've increased our network storage performance significantly for a family of five uses the NAS as the backbone of our digital lives both at home and away on trips.

NVMe support in both M.2 and add-in card (AIC) form factors opens opportunities not available with SATA products. One of the most significant is Intel's Optane technology with a solid 3 to 4x increase in random read performance over existing flash technology. The same NVMe interface also opens the door to larger drives with stronger throughput performance that allows users to enable caching for sequential workloads to take better advantage of 10GbE network speeds.

Today we will use a Synology DS1819+ with a special dual M.2 2280 add-in card with two NVMe SSDs used as a read and write cache. We use Intel's 118GB Optane SSD and Samsung's 1TB 960 Pro flash-based SSD.

Specifications

38

We used the Samsung 960 Pro, an older model that replaced by the new 970 Pro, because we needed two identical drives for the read + write cache test. We 960 Pro allows us to test large capacity MLC flash. The Samsung Pro series is the last from the MLC era, the market has moved on to TLC and even QLC memory technology.

On paper, the Intel 800P Optane shows lower random read performance and that is true, in a way. The 800P delivers higher random read IOPS at low queue depths, but the 960 Pro can scale better at high queue depths.

38

In the chart above we see the random read, write, and 80% mixed workload performance of the 120GB 800P and 1TB 960 Pro. At low queue depths, the 800P is significantly faster with random reads and mixed workloads. The Samsung 960 Pro is only moderately faster with random write performance.

38

What we want to do today is establish what NVMe SSD works better for accelerating a network storage workload. With this information, we can choose the product to use in all of our NAS reviews that don't specifically target the accelerator as the target of the review. For instance, in an upcoming review with a powerful enterprise NAS, we will compare the Intel Optane P4800X and Samsung 983 ZET (Z-NAND SSD). For our standard NAS reviews, we want to simply choose an option from the 800P and 960 Pro, to use in reviews like the upcoming Synology DS1819+.

Samsung 960 PRO 1TB SSD

	Today	Yesterday	7 days ago	30 days ago
	$359.99	$359.99	$359.99	$359.99	Buy
* Prices last scanned on 4/19/2024 at 4:37 am CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission.

CIFS Performance Testing

Testing Notes

38

For this article, we used the Synology DS1819+. This system doesn't support NVMe acceleration out of the box. The system does provide a single PCIe 3.0 x8 slot. With it, you can install a 10bE add-in card or a dual M.2 2280 accelerator card that enables NVMe SSD cache. Sadly, you can't use two cards at the same time with this NAS.

The NAS' configuration software allows us to configure a read-only cache or a read and write cache. The latter requires two drives to ensure data integrity. The read-write cache combines the drives in RAID 1 to mirror the data.

For hard disk drives, we used eight Seagate IronWolf Pro 12TB in RAID 6.

Sequential Read Performance

38

The default cache setting for the Synology DS1819+ ignores sequential data. We unchecked the box to allow the cache drives to work with all data, sequential and random. The bottleneck in this configuration is the 1-gigabit Ethernet that limits throughput.

Sequential Write Performance

38

38

38

Even though the Intel Optane SSD doesn't suffer from steady-state performance loss like flash-based drives, the 960 Pro never drops below the 115 MB/s that is the network bottleneck.

Sequential Mixed Workloads

38

For sequential workloads, NVMe cache acceleration is a wash if your network configuration is limited to 1-gigabit. With all four-cache configurations, we didn't observe any performance or consistency increase over the all HDD array.

Random Read Performance

38

38

The tables turn when we start working with random data. With pure reads, the two Optane tests (read and read + write cache) outperform the other configurations. The two 960 Pro accelerated configurations follow. The pure HDD result is all the way at the bottom of the chart. The furthest gap between the HDD only and the highest cache result shows a difference of around 14,000 IOPS or around a 1400x increase!

Random Write Performance

38

38

38

38

The Synology DS1819+ doesn't deliver a typical random write preconditioning curve like we see with enterprise SSDs. Even with a write cache, the results are very segmented with a lot of inconsistency. The system will use DRAM to cache incoming random write, either from the system itself or on the HDDs.

The highest IOPS we scored on this test were actually from the HDD only configuration, and that's due to the high-speed DRAM that is faster than both flash and 3D XPoint memory. There is a very limited amount of DRAM in place to catch the random writes, so even with the DRAM the results are inconsistent.

Random Mixed Workloads

38

38

The random mixed workload shows gives us our first real difference between the 960 Pro and Optane configuration. At low queue depths, the Optane 800P cache with both read and write acceleration doubles the performance of the 960 Pro read and write acceleration configuration. The results cross over at high queue depths where the 960 Pro takes a significant lead.

Server Workloads

Database Server

38

38

38

38

On this page, we move to server workloads with a 2TB data set. In the database test, we see the read and write configurations score better than the read-only and HDD only configurations. Even though the Optane configuration has nearly 1/10th the cache capacity, it outperforms the 960 Pro read and write configuration.

The read-only cache arrays accelerate performance over the HDD only arrays. The difference is muted compared to what two NVMe SSDs can provide in accelerating the workload.

OLTP Server

38

38

38

38

The OLTP workload shows nearly identical performance as the database test in relation to where the different arrays fall. The read and write cache arrays walk away from the read-only acceleration arrays and all acceleration is better than the HDD only array.

Email Server

38

38

38

38

The email workload uses larger block sizes, and that allows the 960 Pro's to take advantage of its throughput performance. On the steady-state chart, we see that even a single 960 Pro outperforms two Optane SSDs in a read and write acceleration configuration at high queue depths.

Archival File Server

38

38

38

38

The archival file server test is the least know server workload. Dell developed the test for use in the Dell Performance Lab. The workload mixes several block sizes making it a very difficult workload for performance consistency.

In this test, the read-only acceleration outperforms both the HDD only and read/write configurations.

Web Server

38

38

38

38

The web server results favor more cache space over higher speed cache. At the end of the preconditioning run, both the 960 Pro with 1TB of cache, and the Optane array with 120GB cache perform about the same. The ramp up is quite a bit different though.

In the workload chart where we increase the strain on the system through queue depth, we see the Optane read and write configuration outperforming the 960 Pro read and write configuration.

Final Thoughts

Even though NVMe SSD support for cache in storage arrays on "prosumer" grade NAS is a big step, we still have a long way to go. Ideally, these systems will all include high-throughput interfaces like 10-gigabit Ethernet (and beyond) and NVMe acceleration support. The Synology DS1819+ compromises in that aspect, but that's not the only area where it falls short.

38

The add-in card doesn't support M.2 22110 form factor cards, the standard for enterprise M.2 SSDs. With 22110 support, we could have used larger Optane SSDs, or even Samsung 983 DCT drives with hardware power-failure protection. The extra length allows Samsung, and other SSD manufacturers to include power capacitors that purge the data in the DRAM to the flash in the event of a sudden power outage event.

In the future, we should see more support for the M.2 22110 form factor, but as of now, it's a rare feature found only in datacenter and enterprise servers. The drives cost slightly more than consumer models (but available through the channel and Amazon), but designed specifically for these workloads.

To maximize compatibility, we plan to use the Intel Optane SSD 800P drives in our prosumer NAS reviews that support NVMe acceleration. The M.2 2280 form factor will maximize our compatibility until we see the market support the larger form factor. The 800P also reduces the size of data held in the cache and that allows us to look at overall system performance and how the systems work with cache a little better than using very large drives.

Our choice doesn't mean the Optane 800P is the best solution for your business' workload. You can use our results today as a guide to face the right direction. In my own server, I recently replaced an LSI PCI Express SSD with an Intel Optane 900P 480GB drive that works in concert with a Mellanox 40GbE card to distribute data across our overpowered home network with roughly 20 devices randomly pulling data. In my personal testing, we used IMG Burn to read movie data and write back to the NAS a Blu-Ray Ultra (4K) ISO file. The Optane 900P cache configuration allowed the conversion to happen nearly twice as fast as the old LSI cache.

As they say, your mileage may vary. That said, it's difficult to beat 3D XPoint memory technology for low queue depth performance. Intel doesn't specifically list NAS acceleration for these products, but in our testing, the performance benefits are hard to match dollar for dollar.