TweakTown

NVMe NAS Cache: Higher Speed or More Capacity?

NVMe NAS Cache: Higher Speed or More Capacity?

110GB of Intel's Optane memory or 1TB of Samsung's 960 Pro for high-speed NAS cache.

@ChrisRamseyer
Published Fri, Apr 19 2019 10:00 AM CDT   |   Updated Tue, Sep 24 2019 12:28 AM CDT

Introduction, Specifications, and Pricing

More consumer and prosumer-grade network attached storage (NAS) ship with the ability to accelerate workloads with SSDs. We can't pinpoint when the feature rolled out because the software feature was rolled out on existing products and not artificially limited to new models. Chances are, if you own a NAS manufactured in the last four years, the software supports using an SSD cache.

NVMe NAS Cache: Higher Speed or More Capacity? 5 | TweakTown.com
VIEW GALLERY - 38 IMAGES

The latest cache acceleration feature from NAS manufactures is to expand the SSD options to include high-speed NVMe drives. Our QNAP TS-EC1279U-RP first reviewed here in May 2012 added this feature even though the system came to market long before the first NVMe SSD shipped. With NVMe and 40GbE network capabilities, we've increased our network storage performance significantly for a family of five uses the NAS as the backbone of our digital lives both at home and away on trips.

NVMe support in both M.2 and add-in card (AIC) form factors opens opportunities not available with SATA products. One of the most significant is Intel's Optane technology with a solid 3 to 4x increase in random read performance over existing flash technology. The same NVMe interface also opens the door to larger drives with stronger throughput performance that allows users to enable caching for sequential workloads to take better advantage of 10GbE network speeds.

Today we will use a Synology DS1819+ with a special dual M.2 2280 add-in card with two NVMe SSDs used as a read and write cache. We use Intel's 118GB Optane SSD and Samsung's 1TB 960 Pro flash-based SSD.

Specifications

NVMe NAS Cache: Higher Speed or More Capacity? 2 | TweakTown.com

We used the Samsung 960 Pro, an older model that replaced by the new 970 Pro, because we needed two identical drives for the read + write cache test. We 960 Pro allows us to test large capacity MLC flash. The Samsung Pro series is the last from the MLC era, the market has moved on to TLC and even QLC memory technology.

On paper, the Intel 800P Optane shows lower random read performance and that is true, in a way. The 800P delivers higher random read IOPS at low queue depths, but the 960 Pro can scale better at high queue depths.

NVMe NAS Cache: Higher Speed or More Capacity? 3 | TweakTown.com

In the chart above we see the random read, write, and 80% mixed workload performance of the 120GB 800P and 1TB 960 Pro. At low queue depths, the 800P is significantly faster with random reads and mixed workloads. The Samsung 960 Pro is only moderately faster with random write performance.

NVMe NAS Cache: Higher Speed or More Capacity? 4 | TweakTown.com

What we want to do today is establish what NVMe SSD works better for accelerating a network storage workload. With this information, we can choose the product to use in all of our NAS reviews that don't specifically target the accelerator as the target of the review. For instance, in an upcoming review with a powerful enterprise NAS, we will compare the Intel Optane P4800X and Samsung 983 ZET (Z-NAND SSD). For our standard NAS reviews, we want to simply choose an option from the 800P and 960 Pro, to use in reviews like the upcoming Synology DS1819+.

Buy at Amazon

Samsung 960 PRO 1TB SSD

Today Yesterday 7 days ago 30 days ago
$349.77 $349.77 $349.77
Buy
* Prices last scanned on 8/11/2020 at 8:30 pm CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission.

CIFS Performance Testing

Testing Notes

NVMe NAS Cache: Higher Speed or More Capacity? 5 | TweakTown.com

For this article, we used the Synology DS1819+. This system doesn't support NVMe acceleration out of the box. The system does provide a single PCIe 3.0 x8 slot. With it, you can install a 10bE add-in card or a dual M.2 2280 accelerator card that enables NVMe SSD cache. Sadly, you can't use two cards at the same time with this NAS.

The NAS' configuration software allows us to configure a read-only cache or a read and write cache. The latter requires two drives to ensure data integrity. The read-write cache combines the drives in RAID 1 to mirror the data.

For hard disk drives, we used eight Seagate IronWolf Pro 12TB in RAID 6.

Sequential Read Performance

NVMe NAS Cache: Higher Speed or More Capacity? 50 | TweakTown.com

The default cache setting for the Synology DS1819+ ignores sequential data. We unchecked the box to allow the cache drives to work with all data, sequential and random. The bottleneck in this configuration is the 1-gigabit Ethernet that limits throughput.

Sequential Write Performance

NVMe NAS Cache: Higher Speed or More Capacity? 51 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 52 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 53 | TweakTown.com

Even though the Intel Optane SSD doesn't suffer from steady-state performance loss like flash-based drives, the 960 Pro never drops below the 115 MB/s that is the network bottleneck.

Sequential Mixed Workloads

NVMe NAS Cache: Higher Speed or More Capacity? 54 | TweakTown.com

For sequential workloads, NVMe cache acceleration is a wash if your network configuration is limited to 1-gigabit. With all four-cache configurations, we didn't observe any performance or consistency increase over the all HDD array.

Random Read Performance

NVMe NAS Cache: Higher Speed or More Capacity? 55 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 56 | TweakTown.com

The tables turn when we start working with random data. With pure reads, the two Optane tests (read and read + write cache) outperform the other configurations. The two 960 Pro accelerated configurations follow. The pure HDD result is all the way at the bottom of the chart. The furthest gap between the HDD only and the highest cache result shows a difference of around 14,000 IOPS or around a 1400x increase!

Random Write Performance

NVMe NAS Cache: Higher Speed or More Capacity? 57 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 58 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 59 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 60 | TweakTown.com

The Synology DS1819+ doesn't deliver a typical random write preconditioning curve like we see with enterprise SSDs. Even with a write cache, the results are very segmented with a lot of inconsistency. The system will use DRAM to cache incoming random write, either from the system itself or on the HDDs.

The highest IOPS we scored on this test were actually from the HDD only configuration, and that's due to the high-speed DRAM that is faster than both flash and 3D XPoint memory. There is a very limited amount of DRAM in place to catch the random writes, so even with the DRAM the results are inconsistent.

Random Mixed Workloads

NVMe NAS Cache: Higher Speed or More Capacity? 61 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 62 | TweakTown.com

The random mixed workload shows gives us our first real difference between the 960 Pro and Optane configuration. At low queue depths, the Optane 800P cache with both read and write acceleration doubles the performance of the 960 Pro read and write acceleration configuration. The results cross over at high queue depths where the 960 Pro takes a significant lead.

Server Workloads

Database Server

NVMe NAS Cache: Higher Speed or More Capacity? 63 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 64 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 65 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 66 | TweakTown.com

On this page, we move to server workloads with a 2TB data set. In the database test, we see the read and write configurations score better than the read-only and HDD only configurations. Even though the Optane configuration has nearly 1/10th the cache capacity, it outperforms the 960 Pro read and write configuration.

The read-only cache arrays accelerate performance over the HDD only arrays. The difference is muted compared to what two NVMe SSDs can provide in accelerating the workload.

OLTP Server

NVMe NAS Cache: Higher Speed or More Capacity? 67 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 68 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 69 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 70 | TweakTown.com

The OLTP workload shows nearly identical performance as the database test in relation to where the different arrays fall. The read and write cache arrays walk away from the read-only acceleration arrays and all acceleration is better than the HDD only array.

Email Server

NVMe NAS Cache: Higher Speed or More Capacity? 71 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 72 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 73 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 74 | TweakTown.com

The email workload uses larger block sizes, and that allows the 960 Pro's to take advantage of its throughput performance. On the steady-state chart, we see that even a single 960 Pro outperforms two Optane SSDs in a read and write acceleration configuration at high queue depths.

Archival File Server

NVMe NAS Cache: Higher Speed or More Capacity? 75 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 76 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 77 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 78 | TweakTown.com

The archival file server test is the least know server workload. Dell developed the test for use in the Dell Performance Lab. The workload mixes several block sizes making it a very difficult workload for performance consistency.

In this test, the read-only acceleration outperforms both the HDD only and read/write configurations.

Web Server

NVMe NAS Cache: Higher Speed or More Capacity? 79 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 80 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 81 | TweakTown.com
NVMe NAS Cache: Higher Speed or More Capacity? 82 | TweakTown.com

The web server results favor more cache space over higher speed cache. At the end of the preconditioning run, both the 960 Pro with 1TB of cache, and the Optane array with 120GB cache perform about the same. The ramp up is quite a bit different though.

In the workload chart where we increase the strain on the system through queue depth, we see the Optane read and write configuration outperforming the 960 Pro read and write configuration.

Final Thoughts

Even though NVMe SSD support for cache in storage arrays on "prosumer" grade NAS is a big step, we still have a long way to go. Ideally, these systems will all include high-throughput interfaces like 10-gigabit Ethernet (and beyond) and NVMe acceleration support. The Synology DS1819+ compromises in that aspect, but that's not the only area where it falls short.

NVMe NAS Cache: Higher Speed or More Capacity? 100 | TweakTown.com

The add-in card doesn't support M.2 22110 form factor cards, the standard for enterprise M.2 SSDs. With 22110 support, we could have used larger Optane SSDs, or even Samsung 983 DCT drives with hardware power-failure protection. The extra length allows Samsung, and other SSD manufacturers to include power capacitors that purge the data in the DRAM to the flash in the event of a sudden power outage event.

In the future, we should see more support for the M.2 22110 form factor, but as of now, it's a rare feature found only in datacenter and enterprise servers. The drives cost slightly more than consumer models (but available through the channel and Amazon), but designed specifically for these workloads.

To maximize compatibility, we plan to use the Intel Optane SSD 800P drives in our prosumer NAS reviews that support NVMe acceleration. The M.2 2280 form factor will maximize our compatibility until we see the market support the larger form factor. The 800P also reduces the size of data held in the cache and that allows us to look at overall system performance and how the systems work with cache a little better than using very large drives.

Our choice doesn't mean the Optane 800P is the best solution for your business' workload. You can use our results today as a guide to face the right direction. In my own server, I recently replaced an LSI PCI Express SSD with an Intel Optane 900P 480GB drive that works in concert with a Mellanox 40GbE card to distribute data across our overpowered home network with roughly 20 devices randomly pulling data. In my personal testing, we used IMG Burn to read movie data and write back to the NAS a Blu-Ray Ultra (4K) ISO file. The Optane 900P cache configuration allowed the conversion to happen nearly twice as fast as the old LSI cache.

As they say, your mileage may vary. That said, it's difficult to beat 3D XPoint memory technology for low queue depth performance. Intel doesn't specifically list NAS acceleration for these products, but in our testing, the performance benefits are hard to match dollar for dollar.

Buy at Amazon

Samsung 960 PRO 1TB SSD

TodayYesterday7 days ago30 days ago
$349.77$349.77$349.77
* Prices last scanned on 8/11/2020 at 8:30 pm CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission.

Chris Ramseyer started his career as a LAN Party organizer in Midwest USA. After working with several computer companies he was asked to join the team at The Adrenaline Vault by fellow Midwest LAN Party legend Sean Aikins. After a series of shake ups at AVault, Chris eventually took over as Editor-in-Chief before leaving to start Real World Entertainment. Look for Chris to bring his unique methods of testing Hard Disk Drives, Solid State Drives as well as RAID controller and NAS boxes to TweakTown as he looks to provide an accurate test bed to make your purchasing decisions easier.

We openly invite the companies who provide us with review samples / who are mentioned or discussed to express their opinion. If any company representative wishes to respond, we will publish the response here. Please contact us if you wish to respond.

Related Tags

Newsletter Subscription

Latest News

View More News

Latest Reviews

View More Reviews

Latest Articles

View More Articles