Intel Optane in RAID 0 - World's Fastest System Disk

1. Introduction, Drive Specifications, Pricing & Availability
2. Drive Details
3. Test System Setup, RAID Array Setup & Boot Video
4. Synthetic Benchmarks – ATTO & Anvil Storage Utilities
5. Synthetic Benchmarks – CrystalDiskMark & AS SSD
6. Benchmarks (OS) - Vantage, PCMark 7, PCMark 8 & SYSmark 2014 SE
7. Maxed-Out Performance (MOP)
8. Final Thoughts

TweakTown's Rating: 100%

The Bottom Line

With Intel's Optane Memory, you can have the world's fastest OS disk for $240. Enough said!

Introduction, Drive Specifications, Pricing & Availability

82

Intel's "Optane" next-gen phase-change memory is set to forever change the storage industry. What is Optane? Intel calls their 3D XPoint technology "Optane." Optane consists of 3D XPoint memory, Intel memory and storage controllers, Intel interconnect IP and Intel software. 3D XPoint memory is a joint venture between Intel and Micron (IMFT). The actual IP outside of the memory itself is separate proprietary technology. On a cellular level, 3D XPoint is 1000x faster than NAND flash memory. In the real-world, actual performance is limited by bus performance, so even though 3D XPoint is literally an order of magnitude faster than NAND flash, we are limited to about 10x the performance of NAND Flash under the best of circumstances.

82

Optane delivers performance in a different way than NAND flash does. Optane delivers performance where it matters most (random read QD1-4) and exactly where NAND flash is at its weakest (random read QD1-4) as illustrated by the following chart:

82

When evaluating SSDs for purchase, people tend to focus on sequential performance as the primary indicator of how fast an SSD will perform. This is because, over the years, buyers have been trained to look at sequential performance; because that is the way NAND-flash SSDs have been marketed. We want consumers to quit looking at sequential performance and to start looking at low queue-depth random performance, simply because that is what actually matters most in an OS environment. Take a look at the following examples:

82

The sequential numbers circled in red are what consumers always focus on. However, in an OS environment, sequential performance is the least important consideration. What matters is circled in blue; random performance at QD1. Low queue depth random read performance is by far the most important performance metric, with random write at QD1 being the second most important. As you can see with this flash-based array, random read performance is where flash is at its weakest - only 84.77 MB/s or 21,700 IOPS. Now, let's look at an Optane Memory array:

82

Compared to the above flash array, the sequential performance of our Optane Memory array is just a fraction of what the flash array is delivering. Now, look at the numbers circled in blue. These are the numbers that really matter in an OS environment, and this is where Optane is at its best. Look at the difference where it matters most; 4K QD1 random read. Our Optane Memory-based array is cranking out well over 3x the performance of our super-powerful flash-based array; 301.5 MB/s or 77,200 IOPS. This is what makes 3D XPoint memory a game-changing technology, and why it matters to the end-user.

Optane Memory, when used for caching high-capacity conventional storage as intended by Intel, is good stuff, but to us, Optane Memory presents us with a never-before-seen opportunity to create the best performing OS disk the world has ever seen. Optane Memory modules are designed to operate in conjunction with Intel 200 Series motherboards, 7th Gen Intel Core i7 "Kaby Lake" processors and Intel's Rapid Storage Technology (IRST) driver version 15.5 or higher. However, Optane Memory is actually just a small SSD, and it can be used as such. Optane memory doesn't have to be used for caching and is not limited to 200 series or higher Intel motherboards when used as a stand-alone storage device.

Now, of course, Optane Memory does have one glaring drawback. Capacity. Optane Memory comes in two capacities: 16GB and 32GB. It is feasible to use one 32GB Optane Memory module for your OS Disk because as the following screenshot shows:

82

A full clean install of Windows 10 Creators Update X64 is actually only a little over 9GB with all drivers installed on our Z270 system. How did we get it so compact? Simple, we eliminated the hidden hibernation file (open admin CMD prompt and type: "powercfg -h off" then hit enter and restart the system) and turned off the system paging file (virtual memory). Most users do not have a need for either. Even though it is feasible to use a single 32GB Optane memory module, it is a bit too small for most. But, when you RAID two or three together you not only get better performance, you gain enough capacity for most user's OS needs.

To create an OS array with PCIe SSDs, you will need at least two PCIe slots that run through the Intel chipset (PCH). The slots can be either M.2 PCIe or a regular PCIe slot as long as they run through the PCH. Most enthusiast oriented Z170 or Z270 motherboards will accommodate two or three PCIe SSDs running through the chipset. Now that we've explained the feasibility of creating the world's fastest OS disk, let's take a quick look at the Optane Memory module itself and then we will show you how to create the world's best performing OS disk for $240.

Specifications

82

Intel's M.2 x 2280 single sided Optane Memory module is available in two capacities: 16GB and 32GB. The 32GB modules we are benching today sport the following specifications:

• Sequential Read: up to 1,350 MB/s
• Sequential Write: up to 290 MB/s
• Max 4K Random Read Speed: up to 240,000 IOPS
• Max 4K Random Write Speed: up to 65,000 IOPS
• Latency - Read: 9us
• Latency - Write: 30us
• Power - Active: 3.5 Watts
• Power - Idle: 1 Watt
• Endurance: up to 182.5 TBW
• MTBF: 1.6 Million Hours
• UBER: < 1sector per 10^15 bits read
• Reliability feature: End-to-End Data Protection
• Warranty: 5-Year Limited Warranty

The 16GB model retails for $50, the 32GB model $80.

Drive Details

Intel 32GB M.2 x 2280 Optane Memory Module

82

The Optane Memory module ships in a small blue box that says "OPTANE" in the front. The enclosed module's capacity is advertised on this side of the packaging.

82

The back of the packaging advertises the enclosed module's M.2 x 80mm form-factor and its Gen3 x2 interface.

82

Flipping the box open reveals the Optane module is well protected inside of a clear clamshell container.

82

Contained within the packaging is the Optane module, an instruction guide, a warning leaflet and an Optane Memory decal.

82

This side of the module's blue PCB houses the drive's 2-channel ASIC controller, two 16GB Optane Memory packages, and various surface mounted components. We will not be removing the factory label because it has a copper layer inside of it. Removing thermal labels is never a good idea because they bend when removed and will never again lay flat over the components as intended. We will just be providing a photo of the front and back of the module.

82

This side of the memory module is devoid of components.

Test System Setup, RAID Array Setup & Boot Video

Jon's Consumer PCIe SSD Intel Z270 Review Test System - Specifications

• Motherboard: ASRock Z270 Taichi - Buy from Amazon
• CPU: Intel Core i7 7700K @ 5.0GHz - Buy from Amazon / Read our review
• Cooler: Arctic i30 Freezer air cooler
• Memory: Corsair Vengeance LPX DDR4 16GB 3000MHz - Buy from Amazon
• Video Card: Onboard Video
• Case: Corsair Carbide Air 540 - Buy from Amazon / Read our review
• Power Supply: Seasonic Platinum 1000 Watt Modular - Buy from Amazon / Read our review
• OS: Microsoft Windows 10 Professional 64-bit - Buy from Amazon

We would like to thank ASRock, Crucial, Intel, Corsair, RamCity, IN WIN, and Seasonic for making our test system possible.

Creating RAID Array

82

Step one in creating a bootable PCIe storage array is to set storage configuration in BIOS to RAID mode and Storage OpROM Policy to UEFI only. Then reboot. Next, enable RST PCIe Storage Remapping. Then reboot again.

82

Next, create your RAID volume by entering into the Intel Rapid Storage Technology page (located in advanced options). This option will show up in BIOS after completing the first two steps.

82

Click on Create RAID Volume.

82

Select RAID 0 and select the drives you wish to RAID. Intel defaults to 16K stripes, but we prefer 64K stripes. Select stripe size and click on Create Volume.

82

Now you have created your RAID volume. Insert your Windows Installer and reboot.

82

Be sure you are booting your Windows Installer as UEFI when installing Windows. Before installing Windows, you need to grab the latest f6flpy-x64.zip driver, extract it and place it on another USB drive. You can grab it HERE We will be injecting this driver into our Windows install so the array can be seen by the Windows Installer.

82

Once you have the installer to where you select where to install Windows, you will be greeted with no drives in the list. Insert the USB drive with the driver you downloaded. Click on Load Driver. (This is where we inject the driver you downloaded so the array can be seen.)

82

Now click on Browse.

82

Now browse to the f6flpy-x64 you placed on the separate USB drive, and click OK.

82

Hit "Next" to inject the driver. It will take approximately 2 minutes to install the driver.

82

After the driver is installed, Windows Installer can see the array, and away you go. After Windows is installed, install your chipset drivers, and then you need to install the RST control panel/driver to be able to manage the array. You can grab it here. Download the SetupRST.exe file, install it and restart. Now look in your app list under "Intel" and open "Intel Rapid Storage Technology."

82

Click on the "Manage" tab. Expand "Advanced." Disable write-cache buffer flushing, Modify Cache mode to Write back (very important for best performance). Then click on the "Performance" tab and disable Link Power Management (not shown), and restart your PC. Now you are good to go. Don't forget to get rid of the hibernation file as previously mentioned because you will get back a lot of GB's.

82

This is a quick shot of our array as we configured it.

Drive Properties

While we did follow our usual 75% full testing, Optane is different than flash in that performance is not affected by the amount of data on the drive or whether or not the drive is in a "steady-state." TRIM also has no effect on Optane.

2-Drive Optane Array - OS Disk 75% Full

82

3-Drive Optane Array - OS Disk 75% Full

82

All of our testing is performed with our test drive as our boot volume. Our boot volume is 75% full for all OS Disk "C" drive testing to replicate a typical consumer OS volume implementation. We feel that most of you will be utilizing your SSDs for your boot volume and that presenting you with results from an OS volume is more relevant than presenting you with empty secondary volume results.

System settings: Cstates and Speed stepping are both disabled in our systems BIOS. Windows High-Performance power plan is enabled. Windows write caching is enabled, and Windows buffer flushing is disabled. We are utilizing Windows 10 Pro 64-bit OS (Build 15063) for all of our testing except for our MOP (Maxed-Out Performance) benchmarks where we switch to Windows Server 2012 R2 64-bit.

Boot Speed (Restart to Desktop) - Video

This is our 2-drive array. Restart time and back to desktop = 10 seconds. The 3-drive array is even faster.

We will be testing two and three drive Optane Memory arrays. We will only be doing testing with the arrays running as our system (OS) disk. We believe that while our arrays give us plenty of capacity for a system disk, we don't get enough for a secondary storage device. With this in mind, we don't see the need for any of our usual secondary device testing.

Synthetic Benchmarks – ATTO & Anvil Storage Utilities

ATTO

Version and / or Patch Used: 3.05

ATTO is a timeless benchmark used to provide manufacturers with data used for marketing storage products. When evaluating ATTO performance, we focus on the drive's performance curve.

82

82

Even in RAID, Optane Memory doesn't possess strong sequential write performance, but not to worry, because sequential write performance is by far the least important of all performance metrics in an OS environment. We will demonstrate this when we get to our moderate workload testing.

Sequential Write

82

Graphing the performance curve demonstrates the low sequential write performance of Optane Memory. At the other end of the spectrum, we have a pair of 950 Pro's in RAID 0 delivering sequential write performance that nearly maxes out the chipset.

Sequential Read

82

2-Optane modules deliver excellent sequential read performance at low queue depths. 3-Optane modules set the bar for low queue-depth sequential read performance through the PCH. Our 3-drive Optane array is maxing out the chipset at 16KB transfers which is something we've not seen a flash array do.

Anvil Storage Utilities

Version and / or Patch Used: 1.1.0

Anvil's Storage Utilities is a storage benchmark designed to measure the storage performance of SSDs. The Standard Storage Benchmark performs a series of tests; you can run a full test or just the read or write test, or you can run a single test, i.e. 4k QD16. When evaluating performance with Anvils, we focus on total score. When evaluating NVMe SSDs, we are typically looking for a minimum total score of over 10K.

82

82

Scoring

82

Our Optane arrays have no problem easily outscoring the competition because of their otherworldly read performance. You can RAID all the flash based SSDs you want, and they will not be able to come close to the read performance of our Optane Memory arrays.

(Anvil) Read IOPS through Queue Depth Scale

82

82

Astonishing random read performance at QD32. Approaching 500K.

82

This chart perhaps best illustrates what Optane brings to the table. Performance where it matters most in an OS environment. Approximately 70% of all transactions that take place in an OS environment are random reads at low queue depths, with the majority taking place at QD1. At QD 1 both of our Optane Memory arrays are delivering 3-4x the performance of the best that flash has to offer.

(Anvil) Write IOPS through Queue Scale

82

82

Our 3-drive Optane array delivers significantly better random write performance than our 2-drive array.

82

Our Optane arrays may not have good sequential write performance, but they do have what matters most; outstanding random write performance at QD1-2. This is where it is very important for overall system performance to have write-back caching enabled.

Synthetic Benchmarks – CrystalDiskMark & AS SSD

CrystalDiskMark

Version and / or Patch Used: 3.0 Technical Preview

CrystalDiskMark is disk benchmark software that allows us to benchmark 4k and 4k queue depths with accuracy. Note: Crystal Disk Mark 3.0 Technical Preview was used for these tests since it offers the ability to measure native command queuing at QD4. When evaluating CDM results, we focus on 4K random performance at QD1 and QD4.

82

82

82

Focusing in on the all-important 4K and 4K QD4 read performance, we find our Optane arrays absolutely crushing the flash-based contenders. This again illustrates why we are saying you can create the world's fastest OS disk for $240.

82

Both of our Optane arrays deliver better random write performance at QD1 than the competing SSDs in our test pool.

AS SSD

Version and / or Patch Used: 1.8.5611.39791

AS SSD determines the performance of SSDs. The tool contains four synthetic as well as three practice tests. The synthetic tests are to determine the sequential and random read and write performance of the SSD. We evaluate AS SSD performance in terms of overall score. We are looking for a minimum score of 2,000 when evaluating NVMe SSDs

82

82

82

Our Optane arrays don't produce the best overall score, but they do deliver far better read scores than the flash-based competition. Now that we are done with our synthetic testing we can move on to our real-world testing and show you in detail why an Optane Memory array is today's fastest OS disk.

Benchmarks (OS) - Vantage, PCMark 7, PCMark 8 & SYSmark 2014 SE

Moderate Workload Model

We categorize these tests as indicative of a moderate workload environment.

PCMark Vantage - Hard Disk Tests

Version and / or Patch Used: 1.2.0.0

The reason we like PCMark Vantage is because the recorded traces are played back without system stops. What we see is the raw performance of the drive. This allows us to see a marked difference between scoring that other trace-based benchmarks do not exhibit. An example of a marked difference in scoring on the same drive would be empty vs. filled vs. steady state.

Typically, we run Vantage three ways. The first run is with the OS drive 75% full to simulate a lightly used OS volume filled with data to an amount we feel is common for most users. The second run is with the OS volume written into a "Steady State" utilizing SNIA's consumer guidelines. Steady state testing simulates a drive's performance similar to that of a drive that been subjected to consumer workloads for extensive amounts of time. The third run is a Vantage HDD test with the test drive attached as an empty, lightly used secondary device.

With Optane, it doesn't matter what state the drive is in, full, empty or steady-state it runs the same regardless. With this in mind, we are only showing performance at 75% full for comparison.

OS Volume 75% Full - Lightly Used

82

82

82

As we've been telling you, workload performance is primarily driven by low queue-depth random performance. This chart really puts that into perspective. Flash-based workload performance isn't even in the same universe as what we are getting from our Optane Memory arrays. Keep in mind that this is performance with data on the disk.

PCMark 7 - System Storage

Version and / or Patch Used: 1.4.0

We will look to Raw System Storage scoring for evaluation because it's done without system stops and, therefore, allows us to see significant scoring differences between drives. When testing NVMe SSDs on PCMark 7, we are looking for a minimum score of 10,000.

OS Volume 75% Full - Lightly Used

82

82

82

Again, flash offers no competition at all. Keep in mind that Optane Memory is just giving us a fraction of what 3D XPoint can deliver, and even that is overwhelmingly more than flash can do at its fullest potential. I know what you are thinking, what if I had $2600 burning a hole in my pocket and bought 2-2TB 960 Pro's and RAIDed them? Would that beat an Optane Memory array? No, it would not. That will only get you a score of 25K at best.

PCMark 8 - Storage Bandwidth

Version and / or Patch Used: 2.4.304

We use PCMark 8 Storage benchmark to test the performance of SSDs, HDDs, and hybrid drives with traces recorded from Adobe Creative Suite, Microsoft Office, and a selection of popular games. You can test the system drive or any other recognized storage device, including local external drives. Unlike synthetic storage tests, the PCMark 8 Storage benchmark highlights real-world performance differences between storage devices.

OS Volume 75% Full - Lightly Used

82

82

82

PCMark 8 is the most intensive moderate workload simulation we run. With respect to moderate consumer type workloads, this test is what we consider the best indicator of a drive's performance. For us, this really seals the deal. Flash-based SSDs are no longer in the discussion for bleeding-edge enthusiast-level storage performance. These results also serve to illustrate, once again, that in an OS environment low queue depth random read performance is king. We have to look no further than our 950 Pro array to show us why. Notice how the 950 Pro array scores LESS than a single 950 Pro? This is because when you RAID drives to create a boot volume, random read performance at QD1 actually goes down.

82

Again, we want to illustrate how meaningless sequential performance is in an OS environment. This is four 950 Pro 256GB SSDs in RAID 0 delivering 9GB/s of sequential read performance. This is 16-PCIe lanes direct to CPU not routed through a chipset. Now look at the storage bandwidth:

82

Only 883 MB/s storage bandwidth when running consumer workloads. Compare that with the 1,453 MB/s storage bandwidth we are getting from our 3-drive Optane Memory array when running consumer workloads with only 4-PCIe lanes.

BAPCo SYSmark 2014 SE System Performance

Version and / or Patch Used: 2.0.0.70

SYSmark 2014 SE is considered the gold standard for testing system performance because it is an application based benchmark. This test gives us the ultimate in real-world results because it utilizes actual applications running on the system, instead of playing back recorded traces. If you want to know what kind of impact a particular SSD will have on your system's overall performance; this test will show you.

82

82

82

82

Disk performance has the greatest impact on the Responsiveness Score, so that is what we will focus on.

82

Our system is much more powerful than the calibration system (1000-point baseline) used by BAPCo, so we ran an OCZ TL100 120GB SATA III SSD to establish a comparison point relative to our test system. We will be running this test going forward, and we will add drives to our chart as we test them. We decided to chart our results with Server 2012 as well as our results with Windows 10.

It is important to keep in mind that with SYSmark 2014 SE a few points are a big deal when comparing one drive to another on the same platform when running the same operating system. Additionally, real-world application performance is driven primarily by low-queue depth random read performance which is why we are seeing our 2-drive array slightly outperforming our 3-drive array. The 2-drive array has slightly better 4K random read performance at QD1.

The main thing we want point out is that our Optane arrays are scoring well over 100 points more than any flash-based NVMe drive/array is capable of, which is a massive difference in the real-world. Next, we would point out Server 2012 R2 delivers far better real-world disk performance than Windows 10 does.

This caps off our assertion that you can build the world's fastest OS disk for $240 using Intel's Optane Memory modules. BAPCo seems to agree with our findings:

82

TweakTown's score with our 2-drive Optane Memory array is the best BAPCo has ever seen.

Maxed-Out Performance (MOP)

Let's see what our Optane Memory arrays can do with a faster OS. We are utilizing Windows Server 2012 R2 for this testing.

Windows Server 2012 R2

2-Drive Optane Array - OS Disk 75% Full

82

82

82

82

82

82

82

Windows Server 2012 R2

3-Drive Optane Array - OS Disk 75% Full

82

82

82

82

82

82

82

82

Final Thoughts

82

Intel has been developing 3D XPoint for the past ten years, with a vision in mind to revolutionize computing as a whole. Now that day is upon us, and we couldn't be more impressed with the outcome.

As enthusiasts, we are always searching out the fastest hardware and various ways to exploit it for cutting edge performance. Optane Memory modules present us with a never before seen opportunity to create the world's fastest system (OS) disk for about $240. We know some of you will feel that 81GB of usable storage is not enough capacity for your liking. But, as we showed you, an actual full install of the latest version of Windows 10 can be as small as 9GB leaving 70+ gigabytes for software installation, and we believe that is more than enough for most users. We feel that it is more than enough because you should be keeping your games, videos and other large files on a secondary storage device anyway.

As we explained and demonstrated throughout the entirety of this review, 4K random read at QD1-2 has the largest influence on system performance in an OS environment. This is exactly where Optane has changed the game forever. Optane Memory delivers random read performance that is between three and ten times better than the best flash has to offer.

Intel didn't intend for Optane Memory modules to be used for anything other than caching large storage devices, but from the moment we were introduced to Optane Memory at Intel's Folsom California campus, we knew we were going to RAID them.

The user experience that our Optane Memory arrays deliver is unparalleled. The difference Optane brings to the table is palpable; you can really feel it. After using Optane Memory as a stand-alone media in an OS environment, conventional flash-based SSDs just don't provide the rush they once did.

Intel's 32GB Optane Memory Module SSD is TweakTown recommended.

Pros:

• Low QD Random Performance
• Low cost
• Best OS Performance

Cons:

• Low Capacity