LSI MegaRAID SAS3 9361-8i 12Gb/s RAID Controller Review

Introduction

70

LSI flourished during the initial transition to 6Gb/s, and the 9260 paved the way for a period of dominating performance. The LSISAS2108 RoC was revolutionary, and the follow-up LSISAS2208 RoC on the 9265 provided even more dominance. LSI chose to stay with the same 2208 controller on the 9271 as they transitioned to support PCIe 3.0, which left performance roughly the same as the previous generation model. Adaptec, fresh off an acquisition by PMC, rushed in with a radical new 6Gb/s design that boosted native port counts and saturated the PCIe 3.0 bus. This left LSI in a position they are not accustomed to, and Adaptec took the performance crown.

Today, LSI answers as the long-awaited PCIe 3.0 LSISAS2308 makes its debut on the 12Gb/s MegaRAID 9361-8i. The 1.2GHz PowerPC 476 dual-core RoC is paired with 1GB of 1866MHz DDR3 SDRAM. The 9361-8i supports RAID levels 0, 1, 5, 6, 10, 50, and 60 and offers up a spate of strip sizes from 64KB up to 1MB.

The 9361-8i features 8 internal mini-SAS SFF-8643 ports that support up to 240 12Gb/s SAS3 or SATA devices with expanders. The 9361-8i also features backwards compatibility for 6Gb/s and 3Gb/s drives and comes in both 4 and 8 internal port configurations. The 9380-8e offers 8 ports of external connectivity. The 9341 is not outfitted with cache, but it comes with the option of either 4 or 8 internal ports.

The 9361-8i supports LSI's signature Advanced Software Solutions. Fast Path software is included free of charge, and MegaRAID CacheCade Pro 2.0 software is available for purchase. CacheCade Pro allows utilization of SSD volumes as a caching layer to speed up existing HDD infrastructure. SafeStor software creates and manages authentication keys for SED drives to provide an extra layer of security, and CacheVault provides protection for cached data on a flash device with a Battery Backup Unit (BBU).

The 12Gb/s MegaRAID controllers also feature new advanced drive diagnostic technology. In the event of a drive failure, the controller places the drive into a shield state as it performs diagnostics to determine if it is a permanent failure. In many cases, the drive has not actually failed and the MegaRAID controller can bring the drive back online if restoration is possible. This saves precious IT hours, money, and lost compute time from unnecessary RAID rebuild tasks.

One of many attractions of 12Gb/s technology is the doubling of data rates from 24GBps to 48GBps from a 4-lane cable. This enhanced bandwidth can boost existing infrastructure, and LSI provides support for DataBolt bandwidth optimizer technology. This new feature allows two 6Gb/s devices to function over one 12Gb/s connection to maximize ROI.

The 9361-8i's power consumption is 16.15 watts under a drive stress test, while the previous generation 9271-8i required only 11.7 watts. This bump in power consumption is matched with performance gains that should keep the power-per-IOPS threshold within reason. The competing Adaptec ASR-8885 pulls 14.73 watts for a 16-port model.

LSI has integrated Fusion-MPT architecture drivers in every major OS. The software features allow users to customize the solution to their environment, and the continuing evolution of the RoC provides even more speed and flexibility. The 9361-8i comes with a standard three-year warranty.

Today, we test the newest LSI flagship against its formidable competition, the 12Gb/s 16-port ASR-8885 from Adaptec by PMC.

Design and Specifications

LSI 9361-8i Design

70

The 9361-8i features a Low-Profile PCI Express form factor (68.9mm x 167.65mm). The heat sink is a departure from previous LSI designs, and a 70-pin connection for the CacheVault Flash Module resides to the immediate left of the heat sink.

70

Five Micron DDR3 1866 MHz packages line the middle on the rear side of the controller. This totals 1GB of cache in a 72b arrangement (5). The controller also features 16MB of flash ROM for firmware and 32KB of NVSRAM for storing RAID configurations.

70

The rear bracket allows airflow out of the chassis. The controller requires a standard 200 linear feet per minute of airflow.

70

The two SFF-8643 ports feature a horizontal orientation to ease cabling in dense server environments.

70

The cornerstone of the new LSI 12Gb/s controller family is the LSISAS3108 SAS controller, which communicates with the host via a PCIe 3.0 x8 connection.

70

We selected the HGST's SSD800MH SSDs to test performance. Eight of these SSDs are powerful enough to saturate even the fastest of controllers. We have written a product evaluation of the slightly slower version of this SSD, the HGST SSD800MM. Look to these pages soon for an evaluation of the SSD800MH, which features sequential read/write speeds of 1200/750 MB/s and read/write IOPS of 145,000/100,000 IOPS.

LSI 9361-8i Specifications

70

The 9361-8i features a revamped, user-friendly BIOS. LSI also provides other methods to manage the controller. StorCLI provides a command line interface, HII (UEFI Human Interface Infrastructure) handles UEFI interaction, and the easy-to-use MegaRAID Storage Manger manages the card from within the operating system.

Test System and Methodology

70

70

We designed our approach to storage testing to target long-term performance with a high level of granularity. Many testing methods record peak and average measurements during the test period. These average values give a basic understanding of performance but fall short in providing the clearest view possible of I/O Quality of Service (QoS).

'Average' results do little to indicate performance variability experienced during actual deployment. The degree of variability is especially pertinent as many applications can hang or lag as they wait for I/O requests to complete. This testing methodology illustrates performance variability and includes average measurements during the measurement window.

While under load, all storage solutions deliver variable levels of performance. While this fluctuation is normal, the degree of variability is what separates enterprise storage solutions from typical client-side hardware. Providing ongoing measurements from our workloads with one-second reporting intervals illustrates product differentiation in relation to I/O QoS. Scatter charts give readers a basic understanding of I/O latency distribution without directly observing numerous graphs.

Consistent latency is the goal of every storage solution, and measurements such as Maximum Latency only illuminate the single longest I/O received during testing. This can be misleading as a single 'outlying I/O' can skew the view of an otherwise superb solution. Standard Deviation measurements consider latency distribution but do not always effectively illustrate I/O distribution with enough granularity to provide a clear picture of system performance. We utilize high-granularity I/O latency charts to illuminate performance during our test runs.

Our testing regimen follows key SNIA principles to ensure consistent, repeatable testing. The first page of results will provide the 'key' to understanding and interpreting our new test methodology. In replicated environments, RAID 0 can be a compelling choice for bleeding edge performance. RAID 5 provides a layer of data security that protects from the loss of a drive. We test RAID 0 and RAID 5 for this evaluation. During testing, we noted periodic performance irregularities that occurred with both controllers at similar intervals. Due to the similar timing of performance errata with both controllers, we concluded these are GC or other internal functions of the HGST SSD test array.

70

We have found Serialcables.com to be a reliable source of high quality cables and accessories. We have a number of external SAS cables on hand for testing, but none that would fit the internal connectors on the RAID controller. The Dual HD MiniSAS Receptacle (SFF-8644) to Dual Int MD MiniSAS Plug (SFF-8643) allows conversion of internal to external ports. Serial Cables has mainstream, specialty, and limited availability items in stock. When working under deadlines and tight schedules, the fast service and knowledgeable staff at Serial Cables has served us well.

Benchmarks - RAID 0 4k Random Read/Write

70

We precondition the array of eight HGST SSD800MH for 9,000 seconds, or two and a half hours, receiving performance reports every second. We plot this data to illustrate the drives' descent into steady state.

This dual-axis chart consists of 18,000 data points, with the IOPS on the left and the latency on the right. The red dots signify IOPS, and the grey dots are latency measurements during the test. We place latency data in a logarithmic scale to bring it into comparison range. The lines through the data scatter are the average during the test. This type of testing presents standard deviation and maximum/minimum I/O in a visual manner.

Note that the IOPS and Latency figures are nearly mirror images of each other. This illustrates that high-granularity testing can give our readers a good feel for latency distribution by viewing IOPS at one-second intervals. This should be in mind when viewing our test results below. This downward slope of performance only happens during the first few hours of use, and we present precondition results only to confirm steady state convergence.

70

Each level tested includes 300 data points (five minutes of one second reports) to illustrate performance variability. The line for each (Outstanding I/O) OIO depth represents the average speed reported during the five-minute interval. 4k random speed measurements are an important metric when comparing drive performance as the hardest type of file access for any storage solution to master is small-file random. One of the most sought-after performance specifications, 4k random performance is a heavily marketed figure.

The 9361-8i averages 557,037 IOPS, trailing the Adaptec ASR-8885, which scores an average of 685,760 IOPS at 256 OIO. Interestingly, the 9361-8 scores well at lower OIO, nearly doubling the performance of the ASR-8885 at 64 OIO.

70

The ASR-8885 has lower latency at 128 and 256 OIO, but the 9361-8i exhibits much lower latency at lower OIO, staying well under .15ms until 128 OIO.

70

The 9361-8i regains its footing in the random workload with a maximum speed of 578,287 IOPS, while the ASR-8885 trails slightly with an average of 535,766 IOPS at 256 OIO. Once again, we note the enhanced scaling with lower OIO loads from the 9361-8i.

70

The 9361-8i maintains lower latency during write testing.

70

Our write percentage testing illustrates the varying performance of each solution with mixed workloads. The 100 percent column to the right is a pure 4k write workload, and 0 percent represents a pure 4k read workload.

The 9361-8i has a propensity for 4k write workloads. It trails in pure read workloads but takes the lead in heavier write environments.

Benchmarks - RAID 0 8k Random Read/Write

70

70

Server workloads rely heavily upon 8k performance, and we include this as a standard with each evaluation. Many of our server emulations also test 8k performance with various mixed read/write workloads.

The 9361-8i averages 511,493 IOPS at 256 OIO, but the ASR-8885 beats it slightly with 535,358 IOPS.

70

The Adaptec takes the lead in latency performance at higher OIO.

70

The 9361-8i provides an average of 445,752 IOPS at 256 OIO. The ASR-8885 delivers an average of 401,359 IOPS at 256 OIO.

70

70

The 9361-8i closes the gap as we mix in more write activity.

Benchmarks - RAID 0 128k Sequential Read/Write

70

70

128k sequential speed reflects the maximum sequential throughput of the RAID controller. The 9361-8i takes the lead with an average sequential read speed of 5,041 MiB/s. The ASR-885 is close on its heels with an average of 4,872 MiB/s.

70

Latency testing reveals expected scaling.

70

The 9361-8i takes the slightest of leads at 256 OIO with an average of 4,447 MiB/s. The ASR-8885 averages 4,344 MiB/s. Though the competition is stiff at 128 and 256 OIO, the 9361-8i maintains an advantage at lower OIO loadings.

70

70

Write percentage-testing revels both controllers are closely matched with mixed workloads, though the 9361-8i maintains the lead across the board.

Benchmarks - RAID 0 Server Workloads

RAID 0 Database/OLTP

70

70

This test emulates Database and On-Line Transaction Processing (OLTP) workloads. OLTP is the processing of transactions such as credit cards and high frequency trading in the financial sector. Databases are the bread and butter of many enterprise deployments. These demanding 8k random workloads with a 66 percent read and 33 percent write distribution bring even the best solutions down to earth.

The LSI loses slightly at 256 OIO with an average of 431,423 IOPS compared to 438,908 from the ASR-8885. While the 9361-8i loses slightly at the top end, the 9361-8i wins during the remainder of the test (8-128 OIO).

70

RAID 0 Web Server

70

70

The Web Server profile is a read-only test with a wide range of file sizes. Web servers are responsible for generating content users view over the Internet, much like the very page you are reading. The speed of the underlying storage system has a massive impact on the speed and responsiveness of the server hosting the website.

The 9361-8i leads this test convincingly with a top speed of 354,078 IOPS at 256 OIO, while the ASR-8885 averages 309,421 IOPS.

70

RAID 0 Email Server

70

70

The Email Server profile is a demanding 8k test with a 50/50 read/write distribution. This application is indicative of performance in heavy write workloads.

Once again, there is a near-tie at 256 OIO. The 9361-8i averages 439,322 at 256 OIO, and the ASR-885 squeaks past with 442,410 IOPS. The 9361-8i leads the lower 8-128 OIO in both performance and latency.

70

Benchmarks - RAID 5 4k Random Read/Write

70

70

We notice much more differentiation between the two controllers in RAID 5 testing, especially in write workloads. We start with a 4k random read workload that the ASR-8885 leads with an outstanding average of 685,760 IOPS at 256 OIO. The 9361-8i averages 557,037 IOPS at the highest workloads, but it again exhibits solid scaling at the lower-end.

70

70

The 9361-8i tops out at 48,554 IOPS at 128 OIO and then slows down to 45,639 IOPS at 256 OIO. The ASR-8885 reaches its peak of 31,057 at 32 OIO and then drops rapidly to a low of 22,929 at 256 OIO. While both controllers exhibit performance degradation at the highest loading, the LSI weathers the slight dip easily. The ASR-8885 has more profound degradation with heavy write performance scaling.

70

The latency measurements reflect a nearly 2X advantage for the 9361-8i at 256 OIO.

70

Write performance scales as we would expect in this test for the 9361-8i. The ASR-8885 leads with a pure read workload, but it hits a massive cliff when we mix in a light 10 percent write workload. The ASR-8885 cedes the rest of the test to the 9361-8i as we mix in more write activity. Even the slightest of write workloads knocks the ASR-8885 off its perch atop the read chart.

70

We normally do not present some test results, especially if they seem repetitive. We record latency during our mixed workloads, but for simplicities sake, we do not report on 'expected' results.

However, the odd behavior of the ASR-8885 deserves investigation. The LSI plateaus at a 20 percent write workload and is fairly even for the remainder of the test. The ASR-8885 continues to experience increasing latency as we mix in more writes, topping out at a staggering 15.43ms with a 90 percent write workload. We repeated these tests several times to confirm that the results are repeatable.

Benchmarks - RAID 5 8k Random Read/Write

70

70

The 9361-8i tops out at 508,572 IOPS at 256 OIO, while the ASR-8885 reaches 538,099.

70

The 9361-8i provides lower latency at 8-128 OIO, but it loses the lead at 256 OIO.

70

The 9361-8i takes the lead in 8k random write testing with an average of 41,802 IOPS at 128 OIO. The ASR-8885 peaks at 64 OIO with 29,490 IOPS, but it falls significantly under higher load.

70

The 9361-8i provides lower latency during the test.

70

The mixed workload testing shows the cliff both controllers experience when we mix in a 10 percent write workload, though the ASR-8885 falls lower than the 9361-8i. The 9361-8i leads this test with the exception of the pure read workload.

70

We observed the same performance trend from the ASR-8885 in our mixed workload testing. The same latency penalty is incurred as we mix in more write activity, reaching 16ms with a 90 percent write workload.

Benchmarks - RAID 5 128k Sequential Read/Write

70

70

The 9361-8i leads with an average of 5,091 MiB/s in sequential read speed; the ASR-8885 is close behind with 4,910 MiB/s.

70

The 9361-8i provides lower read latency during the test period.

70

The 9361-8i averages 772 MiB/s at 256 OIO, while the ASR-8885 bests it with 852 MiB/s.

70

Latency scales as expected, and the ASR-8885 provides a lower write latency during the test period.

70

Our percentage testing shows the 9361-8i taking the lead in read workloads until we mix in a 30 percent write workload. The ASR-8885 takes the win for the majority of the write workloads in this test.

Benchmarks - RAID 5 Server Workloads

RAID 5 Database/OLTP

70

70

The 9361-8i tops out at 128 OIO with 115,916 IOPS, but it drops slightly at 256 OIO. The ASR-8885 suffers more significant performance degradation from its peak of 82,492 IOPS at 32 OIO.

70

RAID 5 Web Server

70

70

Neither controller exhibits heavy workload performance degradation with this pure-read workload. The 9361-8i takes the lead with 354,291 IOPS at 256 OIO, and the ASR-8885 averages 299,309 IOPS.

70

RAID 5 Email Server

70

70

The 9361-8i closes out our testing with a top speed of 74,450 IOPS at 128 OIO and a slight performance degradation at 256 OIO. The ASR-8885 tops out at 32 OIO with 56,228 IOPS.

70

Final Thoughts

70

The LSISAS3108 12GB/s PCIe RoC is the cornerstone of LSI's new 12Gb/s RAID controller family. The speed increase provides a way forward but also allows performance enhancements with existing infrastructure. Leveraging DataBolt technology allows compatible backplanes and expanders to utilize two 6Gb/s drives per port. The 9361-8i has a great mix of standard features, such as FastPath technology, and a slew of available Advanced Software Solutions that provide customization options.

We address performance with two RAID sets, though a number of types are widely deployed in the datacenter. RAID 0 is attractive for replicated environments that require the utmost speed and can be indicative of both top performance and weaknesses in a RAID controller architecture. RAID 5 provides data protection and is indicative of performance trends with popular nested RAID sets.

During RAID 0 tests, the 9361-8i took second place under heavy 4k and 8k random read workloads, yet it exhibited superior scaling under lighter workloads. The 9361-8i led the charts in 100 percent random write tests, and our mixed workload testing also reflected a preference for random write activity. The 9361-8i led in both sequential read and write workloads. The 9361-8i posted excellent scaling with light workloads in our OLTP and email workloads, but the ASR-8885 took a slight lead at 256 OIO. The 9361-8i easily took the lead in the read-centric webserver workload.

In RAID 5, the gap between the 9361-8i and the ASR-8885 widened considerably. The ASR-8885 led in heavy 4k and 8k 100 percent read workloads, but the 9361-8i exhibited superior scaling at lower OIO. Write percentage testing revealed the slightest of write workloads cause the performance of the ASR-8885 to plummet, giving the 9361-8i the win in 10 of 11 mixed workloads.

70

In 100 percent write workloads, the ASR-885 peaks at 64 OIO and then falls tremendously with heavier workloads, giving the LSI controller a convincing win in random write testing. The ASR-8885's downward performance trends were present in OLTP and email server testing but were absent in read-centric webserver testing. The 9361-8i experienced similar performance degradation at 256 OIO with write workloads, but the trends were not as profound as the ASR-8885, and peak performance was higher.

We found the ASR-8885 exhibits strange latency performance with 4k and 8k mixed workloads. As we mixed in heavier 4k and 8k write workloads, its latency skyrocketed to 15.4ms, while the 9361-8i remained steady at under 6ms. The ASR-8885's RAID 5 performance challenges are hopefully addressable with a firmware update. Adaptec by PMC currently has the lead in native port counts, which can eliminate hardware and power consumption in some situations. It is hard to ignore 16 native ports for dense in-chassis storage deployments.

There is more to characterizing the performance of storage solutions than cracking the whip during a horserace. Measuring winners and losers by maximum performance is simply not possible. Scaling in light workloads is important, and mixed workloads represent the majority of workloads in deployment. The 9361-8i has the finesse to deliver a well-rounded solution for a variety of use-cases and workloads and the brute force to power through difficult parity-based RAID sets. The LSI 9361-8i beats the performance of the ASR-8885 in many key metrics, making it more attractive for external storage and less dense in-chassis designs. The 9361-8i knocks the ASR-8885 off the top of the mountain for now, and its class-leading performance and features merit the TweakTown Editor's Choice Award.

PRICING: You can find the LSI 9361-8i for sale below. The prices listed are valid at the time of writing but can change at any time. Click the link to see the very latest pricing for the best deal.

United States: The LSI 9361-8i retails for $628.70 at Amazon.