KIOXIA CM7-V 3.2TB SSD Review - Gen5 Hits the Datacenter

TweakTown's Rating: 100%

The Bottom Line

A straight 100% rating speaks for itself.

Pros

• + Mixed Workloads
• + Random
• + Sequential

Cons

• - None

Should you buy it?

AvoidConsiderShortlistBuy

Introduction and Drive Details

For the past 6-months or so in the consumer space, PCIe Gen5 SSDs have been redefining storage performance as we know it. With double the potential throughput of PCIe Gen4, PCIe Gen5 has broadened the storage bottleneck allowing for sequential speeds of 14,000 MB/s and 2 million or so 4K random IOPS. In some use-case scenarios, the additional bandwidth can net as much as double the workload performance Gen4 SSDs can muster.

Based on what we've seen so far in the consumer space, we expect a similar leap forward in the enterprise space. Now, as is typical with all storage to this point, more throughput equates to higher power draw, which in turn equates to more heat to mitigate. Additionally, and rarely mentioned, is signal quality. The more bandwidth, the harder it is to keep a reliable signal going from CPU to SSD. Very short traces and/or signal boosters are required for PCIe Gen5 SSDs to operate reliably or even to initialize.

Both heat and signal quality are something we had to overcome on our testing platforms and is worth mentioning here as it is something that enterprise platforms are going to have to deal with as well. The main issue for us on the enterprise side turned out to be U.3 to PCIe adapter related. Fortunately for us, Serial Cables came to the rescue with its U.3 to PCIe Gen5 x4 adapter. As far as we can tell, the key to success seems to be the extremely short traces designed onto this card.

Okay, now, on to the star of today's show KIOXIA's first PCIe Gen5 SSD, the CM7. The CM7-V variant we have on the bench today is a mixed workload specialist capable of up to 14,000 MB/s sequential throughput and up to 2.7 million IOPS random throughput at 3-drive writes per day. The drive comes arrayed with 112L BiCS 5 TLC flash running at what we assume is a speed of 1,600MT.

KIOXIA's CM7 Series SSDs support EDSFF E3.S and 2.5-inch (15mm, z-height) form factors, are designed to NVMe 2.0 / PCIe 5.0 specifications, and are available in two configurations: Read-intensive 1 DWPD with capacities up to 30.72TB (CM7-R), and mixed-use 3 DWPD with capacities up to 12.80TB (CM7-V). The CM7 series includes features like a dual-port design for High Availability (HA) applications, flash die failure protection that maintains reliability in the cases of die failure, and a Self-Encrypting Drive (SED) that supports TCG Opal and TCG Ruby, as well as a SED option of FIPS 140-35.

Now, before we get into our regular test suite, we want to mention that the 3.2TB CM7-V we are reviewing today is factory rated for up to 2.7 million IOPS at QD256 running on a Linux -based platform. We mention this because part of what we do here is verify that our test subject can indeed deliver quoted up-to-performance metrics. With our normal test configuration, we were able to exceed all stated up-to-performance metrics except for 4K random read, where we peaked at 2.26 million IOPS at QD256.

Our normal test configuration is Windows Datacenter 2022 based using 8-CPU threads, and until this point, we've never had an issue hitting quoted up-to-performance metrics with this configuration. So, what to do? Well, we decided to give it a go with 16-CPU threads at QD512, and voila!

34

2.81 million IOPS, which does indeed verify the 3.2TB CM7-V can deliver advertised random throughput even on a Windows Server-based platform. Impressive is an understatement. 112-Layer flash delivering 2.8 Million IOPS? Wow!

We wanted to show this upfront because we won't be changing our normal test configuration of 8-CPU threads at up to QD256 because there is no compelling reason for us to do so. QD256 is well beyond where even datacenter SSDs typically run workloads at. Additionally, 8-CPU threads vs. 16-CPU threads at QD128 proved to be identical at 1.5 million IOPS. In short, we are of the considered opinion that our normal test configuration is plenty sufficient to replicate real-world performance even for PCIe Gen5 SSDs.

Specs/Comparison Products

34

34

The CM7-V 2.5" U.3 model is offered at four capacity points and four variants at each capacity point. We find it particularly impressive that the drive can deliver as it does within a 25-watt power envelope, as even some PCIe Gen4 SSDs will draw 25 watts.

KIOXIA CM7-V 3.2TB SSD NVMe PCIe Gen5 x4 U.3 SSD

Enterprise Testing Methodology

TweakTown strictly adheres to industry-accepted Enterprise Solid State Storage testing procedures. Each test we perform repeats the same sequence of the following four steps:

1. Secure Erase SSD
2. Write the entire capacity of SSD a minimum of 2x with 128KB sequential write data, seamlessly transition to the next step
3. Precondition SSD at maximum QD measured (QD32 for SATA, QD256 for PCIe) with the test-specific workload for a sufficient amount of time to reach a constant steady-state, seamlessly transition to the next step
4. Run test-specific workload for 5-minutes at each measured Queue Depth, and record results

Benchmarks - Random and Sequential

4K Random Write/Read

34

We precondition the drive for 16,000 seconds, receiving performance data every second. We plot this data to observe the test subject's descent into steady-state and to verify steady-state is in effect as we seamlessly transition into testing at queue depth. Steady-state is achieved at 2,000 seconds of preconditioning. Average steady-state write performance at QD256 is approximately 635K IOPS.

34

34

We hit a max of 637K IOPS sustained, or 6% higher than factory up to specs of 600K IOPS. Comparing the CM7-V to its predecessor, the CM6-V, the CM7-V is capable of up to 90% more performance with a pure 4K random write workload than its PCIe Gen4 predecessor. Impressive.

34

34

As we explained previously, when using 8-CPU threads at QD256, we top out at 2.26 million 4K random read IOPS, but as also shown previously, the drive is capable of up to at least 2.8 million IOPS. At the 2.26 million IOPS shown here, the CM7-V 3.2TB bests our previous champion, the 7.68TB 9400 Pro, by 582K IOPS. Compared with its predecessor, the CM6-V, our test subject is 63% more performant at QD256.

8K Random Write/Read

34

We precondition the drive for 16,000 seconds, receiving performance data every second. We plot this data to observe the test subject's descent into steady-state and to verify steady-state is in effect as we seamlessly transition into testing at queue depth. Steady-state is achieved at 2,000 seconds of preconditioning. Average steady-state write performance at QD256 is approximately 335K IOPS.

34

34

We expect 8K random to track exactly the same as 4K random here, just at a lower rate because it's moving twice the amount of data. Just as we witnessed at 4K, the CM7-V is serving up massive performance at every queue depth. Nothing else is even close. Double the performance of its predecessor, the CM6-V. QD1 performance here is most impressive, even more so than at 4K.

34

34

As good as it delivers at 4K, the CM7-V delivers even better at 8K, setting new lab records at every measured queue depth. Outstanding.

128K Sequential Write/Read

34

We precondition the drive for 6,500 seconds, receiving performance data every second. Steady-state for this test kicks in at 0 seconds. Average steady-state sequential write performance at QD256 is approximately 7,100 MB/s.

34

34

KIOXIA specs our test subject as capable of up to 6750 MB/s sequential 128K writes. We are getting a lot more. Again, the drive impresses most at QD1, where it's beating the second-place CM6-V by more than 2,700 MB/s. As we climb the queue depth ladder, our test subject again sets a new lab record at each rung.

34

34

14,000 MB/s at QD32 is what our test subject is rated for, and that seemed a pretty lofty claim when we first saw it. However, even this claim turns out to be a bit on the conservative side as we are able to exceed 14,500 MB/s at QD16. Incredible.

Benchmarks - Workloads

4K 7030

4K 7030 is a commonly quoted workload performance metric for Enterprise SSDs.

34

We precondition the drive for 16,000 seconds, receiving performance data every second. We plot this data to observe the test subject's descent into steady-state and to verify steady-state is in effect as we seamlessly transition into testing at queue depth. Steady-state is achieved at 4,000 seconds of preconditioning. Average steady-state performance at QD256 is approximately 1,046K IOPS.

34

34

Mixed workload performance is performance that matters. Again, our test subject sets a new standard at every measured queue depth culminating in 22% more performance than we've experienced to date at QD256.

Email Server

Our Email Server workload is a demanding 8K test with a 50 percent R/W distribution. This application gives a good indication of how well a drive will perform in a write-heavy workload environment.

34

We precondition the drive for 16,000 seconds, receiving performance data every second. We plot this data to observe the test subject's descent into steady-state and to verify steady-state is in effect as we seamlessly transition into testing at queue depth. Steady-state is achieved at 3,000 seconds of preconditioning. Average steady-state performance at QD256 is approximately 420K IOPS.

34

34

Now that we are back into 8K aligned territory, the CM7-demonstrates even more performance superiority at low queue depths than at 4K. 14% more at QD1 than the next best and 22% better than the second best at QD256.

OLTP/Database Server

Our On-Line Transaction Processing (OLTP) / Database workload is a demanding 8K test with a 66/33 percent R/W distribution. OLTP is online processing of financial transactions and high-frequency trading.

34

We precondition the drive for 16,000 seconds, receiving performance data every second. We plot this data to observe the test subject's descent into steady-state and to verify steady-state is in effect as we seamlessly transition into testing at queue depth. Steady-state is achieved at 4,000 seconds of preconditioning. Average steady-state performance at QD256 is approximately 500K IOPS.

34

34

As expected by this point, our test subject again delivers more than we've ever experienced at every measured queue depth. 10% more at QD1 and 21% more at QD256. A 500K IOPS database SSD? Yes, please.

Web Server

Our Web Server workload is a pure random read test with a wide range of file sizes, ranging from 512B to 512KB at varying percentage rates per file size.

34

We precondition the drive for 16,000 seconds, receiving performance data every second. We plot this data to observe the test subject's descent into steady-state and to verify steady-state is in effect as we seamlessly transition into testing at queue depth. We precondition for this test with an inverted (all-write) workload, so no relevant information can be gleaned from this preconditioning other than verification of steady-state.

34

34

We consider this test to be the most taxing test we run, and as such, we find results here to be the most impressive we've ever experienced at any time ever. Our test subject is delivering 2.65x the performance of its predecessor at QD256 and a whopping 29% more than the next best. Amazing really.

Final Thoughts

As we remember things, KIOXIA was the first to demonstrate operational PCIe Gen4 SSDs, and also the first to demonstrate operational Gen5 SSDs. Now whether or not our memory serves us correctly, one thing is for sure, KIOXIA is always positioned at the forefront of the enterprise storage industry.

KIOXIA's CM7 series is a perfect example of not only being first but doing things right. The drive didn't give us any thermal-related issues and behaved exactly as expected, even down to the smallest of details, such as Secure Erasing. It's rare indeed for any enterprise SSD to behave this well.

34

And then there is the performance, which is as we see it - mind-blowing, especially considering we could extract 2.81 million IOPS from BiCS 5 112-Layer flash. We didn't think that would be possible, but yep, there it is, and again another result that speaks to KIOXIA's architectural competence. KIOXIA's CM7-V is by far the most performant SSD we've ever tested and, overall, the most efficient as it relates to performance per watt. Editor's Choice.