CoreRise Comay BladeDrive E28 1.6TB Enterprise PCIe SSD Review

Introduction

The CoreRise Comay BladeDrive E28 PCIe SSD comes in a full-height 3/4-length form factor. The BladeDrive is a sequential access dynamo, with top speeds of 3,400 MB/s sequential read and 3,200 MB/s sequential write. 4k random read weighs in at an impressive 520,000 IOPS, and 4k random write speed is 440,000 IOPS. Perhaps most impressive are the massive capacity points available. The model we are testing today comes with 1,600GB of storage, and models are available with 800GB and 3,200GB.

Comay is the SSD division for the China-based company CoreRise. Comay is one of the world's largest manufacturers of SSDs and features an extensive line of SSDs for enterprise, industrial, military, and client environments. Comay invests considerable resources into R&D, and their broad background and manufacturing capability leads to many unique features and builds on their SSD products.

Popular Now: Grand Theft Auto 6 release date confirmed by Take-Two

Comay's BladeDrive is their first foray into the PCIe SSD market. Comay's experience with multiple custom LSI SandForce SSDs went into the design and build of the BladeDrive E28. The BladeDrive crams eight of the LSI SF-2581 controllers onto the drive. The SF-2581 is an enterprise grade controller with extensive history in the datacenter, and the BladeDrive marks the debut of Comay's first RAID controller. The SBC208 is an 8-channel proprietary controller Comay designed internally, and it is used exclusively in their BladeDrive PCIe SSDs.

Comay utilizes their Cap-X system for power fail protection. Two large capacitors provide enough power to flush the data to the NAND in the event of power loss. LSI SandForce controllers do not utilize DRAM, which is a significant advantage in power loss situations.

Power consumption can be prohibitive with some large capacity PCIe SSDs. The Comay BladeDrive consumes a maximum of 35W under load, which is over the 25W technical limit on some motherboards. Many motherboards can easily handle 35W from the PCIe slot, and unlike other solutions, the BladeDrive E28 doesn't require external power. This reduces cabling and clutter.

Data protection is handled with AES-128, and the BladeDrive also supports SCSI UNMAP commands. This provides TRIM functionality in situations where the host operating system can pass the command. LSI SandForce RAISE technology (Redundant Array of Independent Silicon Elements) provides RAID-5 like data protection, and BCH ECC is rated to 55 bits per sector.

Making a jump into the PCIe space can be a bit daunting; there are several entrenched players in this space with a long history of delivering reliable solutions. Today, we are placing the BladeDrive against the Micron P420m and the Micron P320h. The P420m is a value-oriented PCIe SSD that utilizes 25nm MLC NAND, while the P320h sets the bar for high performance with its SLC NAND. This varied test pool will help ascertain the position of the BladeDrive in the PCIe SSD market.

A considerable strength of the BladeDrive comes in its capacity. There simply are not many PCIe SSDs that offer 3,200GB of flash capacity on one device. The BladeDrive also provides endurance ratings of 3, 6, and 12 PiB. The endurance threshold is competitive with other MLC PCIe SSDs on the market.

The design utilizes SSD and RAID controllers to handle all processing. This is in contrast to some solutions that use the host system resources for SSD management. When scaling beyond one PCIe SSD per system, the offloaded architecture allows for cleaner performance scaling and does not subject the host system to undue resource consumption. Comay provides multi-drive performance specifications that indicate near-linear performance scaling.

Comay also includes an easy-to-use GUI for management and SMART data monitoring. The BladeDrive E28 features an MTBF of 2,000,000 hours and a 3-year limited warranty. Let's take a look at the impressive collection of flash on the next page.

Design and Specifications

Comay BladeDrive Design

The Comay BladeDrive E28 sports an impressive profile. The SSD comes in a full-height 3/4-length PCIe form factor. The SSD we evaluated is a pre-production model, but we noted some bowing in the center of the card. The thermal pads employed between the two PCBs were a bit too thick, but this will likely be addressed in production runs. We can see the tiered design of the card with four layers of components. A large heat sink covers the top of the card, shedding heat into the server chassis.

A large rear covering doubles as the baseplate for the drive. The perforated rear bracket facilitates the required 300LFM airflow.

Four large thermal pads stretch the length of the bottom of the heat sink to cool the uppermost four rows of NAND. A smaller heat sink to the left covers the RAID controller.

Peeling the SSD apart reveals two connectors that mate the PCBs together. Sixty-four Toshiba 19nm MLC NAND packages line this side of the PCBs.

An additional 64 NAND packages line the rear of the PCBs for a total of 128 TSOP NAND emplacements on the BladeDrive. Each NAND package holds 16GB. This gives the SSD 2,048GiB of RAW NAND. The PCBs also provides mount points, hidden under the NAND die, for BGA packaged NAND.

Eight SF-2581 controllers are spread amongst the PCBs, with each controlling 16 NAND packages. Each SSD controller handles 256GB of RAW NAND, 200GB of which is user-addressable. The remaining capacity of each SSD is split between overprovisioning and RAISE functionality (Redundant Array of Independent Silicon Elements). RAISE provides RAID 5-like redundancy to each individual bank of NAND controlled by the SF-2581 controllers. These SSDs are configured in RAID 0 for optimum performance and maximum capacity.

The Comay SBC208-2 controller aggregates the eight SF-2581 controllers and presents them to the operating system as one large striped volume. The SBC208 controller is an 8-channel proprietary RAID controller that is developed and used exclusively for the BladeDrive PCIe SSD. To the bottom right, we notice two supercapacitors that provide enough power to flush the data to the NAND in the event of power loss. SandForce SSDs do not require DRAM. This provides them an inherent advantage in power-loss scenarios.

Comay BladeDrive Specifications

The Comay BladeDrive E28 supports Windows Server 2012 and 2008, along with all major Linux distributions. The unrecoverable bit error rate is less than 1 sector per 10^16 bits read. DuraWrite provides compression capabilities to minimize NAND wear, and intelligent global dynamic and static wear-leveling also enhances flash life.

Test System and Management

Test System and Methodology

We designed our approach to storage testing to target long-term performance with a high level of granularity. Many testing methods record peak and average measurements during the test period. These average values give a basic understanding of performance, but fall short in providing the clearest view possible of I/O Quality of Service (QoS).

'Average' results do little to indicate performance variability experienced during actual deployment. The degree of variability is especially pertinent as many applications can hang or lag as they wait for I/O requests to complete. This testing methodology illustrates performance variability, and includes average measurements, during the measurement window.

While under load, all storage solutions deliver variable levels of performance. While this fluctuation is normal, the degree of variability is what separates enterprise storage solutions from typical client-side hardware. Providing ongoing measurements from our workloads with one-second reporting intervals illustrates product differentiation in relation to I/O QoS. Scatter charts give readers a basic understanding of I/O latency distribution without directly observing numerous graphs.

Consistent latency is the goal of every storage solution, and measurements such as Maximum Latency only illuminate the single longest I/O received during testing. This can be misleading as a single 'outlying I/O' can skew the view of an otherwise superb solution. Standard Deviation measurements consider latency distribution, but do not always effectively illustrate I/O distribution with enough granularity to provide a clear picture of system performance. We utilize high-granularity I/O latency charts to illuminate performance during our test runs.

Our testing regimen follows key SNIA principles to ensure consistent, repeatable testing. LSI SandForce's inline compression engines give performance a boost with compressible data. Many workloads, including OLTP and database workloads, are highly compressible. Other workloads have varying levels of compressibility. In the interest of representing the broadest swath of use-cases, we test with 50 percent compressible data. The first page of results will provide the 'key' to understanding and interpreting our new test methodology.

Management

The Comay BladeDrive E28 comes with a comprehensive software suit to monitor and manage the SSD. Users can monitor endurance and temperature, and can even secure erase the drive. The GUI also offers basic benchmarking capabilities and partition management. The GUI is not as polished as other solutions but is clearly a work in progress. This inaugural management utility for their new series of PCIe SSDs is a good start.

Benchmarks - 4k Random Read/Write

4k Random Read/Write

We precondition the Comay BladeDrive E28 for 9,000 seconds, or two and a half hours, receiving performance reports every second. We plot this data to illustrate the descent into steady state.

This dual-axis chart consists of 18,000 data points, with the IOPS on the left and the latency on the right. The red dots signify IOPS, and the grey dots are latency measurements during the test. We place latency data in a logarithmic scale to bring it into comparison range. The lines through the data scatter are the average during the test. This type of testing presents standard deviation and maximum/minimum I/O in a visual manner.

Note that the IOPS and Latency figures are nearly mirror images of each other. This illustrates that high-granularity testing can give our readers a good feel for latency distribution by viewing IOPS at one-second intervals. This should be in mind when viewing our test results below. This downward slope of performance only happens during the first few hours of use, and we present precondition results only to confirm steady state convergence.

Each level tested includes 300 data points (five minutes of one second reports) to illustrate performance variability. The line for each OIO depth represents the average speed reported during the five-minute interval. 4k random speed measurements are an important metric when comparing drive performance as the hardest type of file access for any storage solution to master is small-file random. One of the most sought-after performance specifications, 4k random performance is a heavily marketed figure.

The Comay BladeDrive averages 314,645 IOPS at 256 OIO. The Micron P320h leads with 788,071 IOPS, followed closely by its little brother, the Micron P420m, at 728,111 IOPS at 256 OIO.

The Comay falls short of its specified 520,000 random read IOPS. It is important to note we achieved the rated top speed with FOB (Fresh Out of Box) testing, and with 100 percent compressible data, the SSD scores much higher.

The BladeDrive E28 provides a nearly linear latency scaling as we add in heavier workloads, but at 256 OIO, variability increases. The Micron SSDs offer cleaner latency scaling as the workload increases.

The BladeDrive E28 provides an average of 205,105 IOPS at 256 OIO. The P420m provides an average of 108,342 and the P320h delivers an average of 205,376 IOPS.

The P320h features SLC NAND, so comparisons aren't entirely apples to apples. The E28 delivers an impressive average, and though it experiences variability, it is well above the P420m in performance in this test. The BladeDrive is spec'd at 440,000 IOPS, and with varying levels of compressibility, its peaks are much higher than this 50 percent compressible workload.

The BladeDrive E28 encounters significant variability with the 4k random workload. The P420m and BladeDrive E28 experience variability in this heavy write workload, while the P320h stays steady, a benefit of its SLC NAND. The BladeDrive wins this test in the MLC segment.

Our write percentage testing illustrates the varying performance of each solution with mixed workloads. The 100 percent column to the right is a pure 4k write workload, and 0 percent represents a pure 4k read workload.

The BladeDrive exhibits a nice middle region in performance, and it provides solid performance with write-centric workloads in this test.

Benchmarks - 8k Random Read/Write

8k Random Read/Write

Server workloads rely heavily upon 8k performance, and we include this as a standard with each evaluation. Many of our server emulations also test 8k performance with various mixed read/write workloads.

The Comay BladeDrive E28 delivers an average of 211,724 8k random read IOPS at 256 OIO, while the Micron P320h delivers 406,167 IOPS and the Micron P420m delivers an impressive 387,785 IOPS.

The BladeDrive has the highest latency recorded during the test.

The BladeDrive excels with heavy random write workloads. The BladeDrive averages 138,206 IOPS at 256 OIO, while the P320h provides 103,124 IOPS and the P420m weighs in at 54,523 IOPS at 256 OIO. The BladeDrive beats the SLC powered P320h in this test, which is quite the feat. There is variability during the test, but most of the performance is well above the competing Micron drives.

The Comay takes the lead in the latency test, and the P420m experiences some variability.

The BladeDrive trails from 0-50 percent, but retakes the lead as we mix in heavier write workloads.

Benchmarks - 128k Sequential Read/Write

128k Sequential Read/Write

128k sequential speed reflects the maximum sequential throughput of the SSD. The Comay BladeDrive E28 scales well as we mix in more outstanding I/O, but the Micron SSDs stay at high throughput even at lower OIO. The BladeDrive tops out at an impressive 2,982 MiB/s at 256 OIO, whereas the Micron P320h averages 3,064 and the Micron P420m leads with an average of 3,336 MiB/s.

Comay indicates the BladeDrive can reach a top speed of 3,600 MB/s with 2MB and 8MB data.

The BladeDrive provided the lowest latency during the test.

The BladeDrive runs away with this test with a fantastic average of 2,799 MiB/s. The P320h musters 1,904 MiB/s of write throughput, and the P420m trails with 613 MiB/s. Heavy sequential write workloads exhibit the BladeDrive's affinity for write workloads.

The Comay BladeDrive provides the lowest latency during the measurement window.

Benchmarks - OLTP - Web Server Workloads

Database/OLTP

This test emulates Database and On-Line Transaction Processing (OLTP) workloads. OLTP is the processing of transactions such as credit cards and high frequency trading in the financial sector. Databases are the bread and butter of many enterprise deployments. These demanding 8k random workloads with a 66 percent read and 33 percent write distribution bring even the best solutions down to earth.

The Comay bests the Micron P420m with its overall average of 125,981 IOPS at 256 OIO. Intense variability muddies the picture for the E28. The Micron P320h averages 189,814 IOPS, and the Micron P420m delivers pristine performance consistency with an average of 106,403 IOPS.

Web Server

The Web Server profile is a read-only test with a wide range of file sizes. Web servers are responsible for generating content users view over the Internet, much like the very page you are reading. The speed of the underlying storage system has a massive impact on the speed and responsiveness of the server hosting the website.

The BladeDrive performs well in this read-only workload with an average of 178,019 IOPS setting slightly above the 171,511 IOPS from the P320h. The Micron P420m steals the show with an average of 188,438 IOPS.

Benchmarks - Email Server Workload

Email Server

The Email Server profile is a demanding 8k test with a 50/50 read/write distribution. This application is indicative of performance in heavy write workloads.

The BladeDrive averages 95,887 IOPS, but intense variability produces many operation lags that are crippling to application workloads. The P320h averages 168,575 IOPS at 256 OIO, and the P420m provides a very consistent 87,735 IOPS.

Final Thoughts

CoreRise, and their SSD-wing Comay, have made significant inroads into the SSD arena in a relatively short time span. In fact, pumping out 60,000 SSDs per month may make Comay the largest SSD supplier many people aren't familiar with. Wading into the PCIe SSD realm is an obvious progression from their roots in commercial, industrial, enterprise, and OEM markets.

Comay certainly didn't make a shy entrance; their 3,200GB behemoth is designed to provide maximum flash density for cloud computing, database, web hosting, video on demand, and other enterprise storage applications. Being a relative unknown in the PCIe space puts the responsibility on Comay to deliver reliability and trust in their products.

The BladeDrive E28 utilizes the proven SF-2581 controllers, and RAISE protects against page or block failure for each individual SSD on the device. The BladeDrive utilizes RAID 0 to maximize performance and capacity of all 8 drives, but we would like to see an option to utilize RAID 5 for more protection from data loss. Scaling of multiple devices is optimal, so RAID 1 configurations with multiple PCIe SSDs remain an option. Unfortunately, for many deployments, that wouldn't be a financially viable solution.

The new Comay SBC208 RAID controller is a great first entrant from Comay. It handles performance and keeps latency during most workloads within expectations. Many of the performance trends we noted are familiar hallmarks of LSI SandForce controllers. The Comay SBC208 handles demanding workloads well. Our test suite is demanding by design, and we executed it numerous times over several months. The BladeDrive performed well and had no compatibility issues on our platforms. The SBC208 seems to be a polished product already, but there will surely be more enhanced versions in the future.

In our testing, the BladeDrive E28 was surprisingly resilient in heavy write workloads. We tested with 50 percent compressible data, which is a tradeoff designed to pander to both ends of the compressibility spectrum. The nature of LSI SandForce controllers provides more performance with compressible data, so there is a possibility of higher performance with many inherently compressible workloads.

We didn't pull any punches and tested the BladeDrive against two heavyweight contenders. The Micron P420m and P320h are top-class solutions. The P320h defines the upper limit of performance from a PCIe SSD from its SLC NAND, and the P420m utilizes more economical MLC NAND for a value alternative. The BladeDrive lagged behind the Micron SSDs in random read performance, but provided faster random write performance, albeit with some variability.

The BladeDrive E28 excels in heavy write workloads, with much of our percentage testing revealing great write performance. The Micron SSDs provided better performance in our transactional server workloads, but the BladeDrive provided outstanding sequential performance. Sequential read and write topped the charts with heavy workloads. Video-on-demand and other sequential workloads would benefit from this type of solution.

The primary reason to deploy flash is for increased performance density and economic power consumption. The CoreRise Comay BladeDrive E28 PCIe SSD delivers on both counts, and if priced aggressively, it can generate quite a bit of traction, particularly in the exploding APAC datacenter segment. Pricing will be important to compete with the foundries, but not all NAND fabricators are shipping PCIe SSDs in volume. The focus from Comay on software management software also highlights their commitment to providing an easy-to-use solution. Some PCIe SSD manufacturers have more rudimentary interfaces.

Overall, this is a great first step from CoreRise into the PCIe SSD market, and for its management software and high capacity point, we award the BladeDrive E28 the TweakTown Best Features Award.