Serial Cables 24-bay 12Gbps SAS Enterprise JBOD Enclosure Review

TweakTown's Rating: 91%

The Bottom Line

The Serial Cables SA-ENC12G-024A is constructed of quality components that are very durable. The JBOD provides unhindered performance to any SATA or SAS device used in the enclosure, and supports dual-port connections as well. Models are available with expanders as well.

Introduction

55

Serial Cables has a history of providing a wide range of cabling and specialty items, including fibre channel (cables and FSP's), HD SAS/SATA, SCSI, PCIe, and Infiniband cabling. Serialcables.com also supplies adaptors and accessories, along with wiring diagrams and pictures of all the products they supply. Serial Cables is a certified LSI Channel Alliance Partner and SandForce Trusted Partner, and they are the exclusive North American distribution partner for SerialTek.

Serial Cables provided their inaugural product geared for lab environments and testing applications with their eight-bay 12Gb/s JBOD. In our Serial Cables SA-ENC12G-01A 12Gb/s SAS3 JBOD External Storage Review, we found the JBOD offers a solid platform with quality components.

The success of the eight-port model led to development of larger 24-bay models. Serial Cables is expanding their presence in the market by offering their new SA-ENC12G-024A 24-port Direct Attach JBOD. The JBOD is specifically designed to meet the needs of 2.5" HDD/SSD testing environments, but also works well in production environments, particularly when paired with an expander, or when purchased with optional internal expanders. 12Gb/s SAS requires low-loss and high-quality PCB assemblies due to a requirement for enhanced transmit and receive equalization. The Direct Attach JBOD satisfies these requirements with quality components and internal cabling. The enclosure comes in a 19" 2U rack-mountable form factor.

Serial Cables also offers a larger 24-bay SAS/SATA test box that features individual, and replaceable, interposers. This larger version also offers individual drive power control, and current measurement, via CLI with USB, Telnet, or Ethernet connections. That enclosure provides 800 watts of power, and comes in a rack-mountable 19" configuration.

The SA-ENC12G-024A we are evaluating today is available in two configurations. As a direct attach model there are no expanders, and each drive is accessible via dual-port connections. There is also the option of 48-port 12Gb/s SAS expanders for storage environments requiring expanded high-availability features, and customers can tailor the device for their needs by selecting either one or two 12Gb/s LSI SAS 3x48 expanders for the chassis.

The 12Gb/s backplane leverages SFF-8680 connectors and SFF-8643 internal HD MiniSAS receptacles. For external connectivity, the JBOD provides 12 SFF-8644 HD MiniSAS receptacles. Each drive is addressable in single-port or dual-port mode.

A notable step forward from the eight-port model is the redundant PSUs featured on the 24-port model. These feature two 38X38X28 DC fans per module, and the 48-pin modules are hot-swappable/hot-pluggable. Each module also features its own AC inlet, and is rated for a total output power of 500 watts.

The JBOD features plenty of airflow, along with a rugged design that is well suited for test environments. The Intel lab recently conducted testing for the Intel RAID Performance 12Gb/s SAS RAID Controllers whitepaper with the eight-bay and 24-bay Serial Cables JBODs. From performance characterization to device verification, and even production environments, the Serial Cables Direct Attach JBOD is designed to meet the needs of demanding users. Let's load it up with an array of 12Gb/s SAS SSDs, and test its performance.

SA-ENC12G-024A Internals and Specifications

SA-ENC12G-024A Internals

The JBOD comes in a rack-mountable 19" 2U chassis with a solid construction. There are 24 ports for storage devices, and fan and power-fail notification lights near the right handle. A reset alarm button silences the alarm.

55

The rear of the chassis reveals 12 SFF-8644 HD MiniSAS receptacles. Each dual set of connectors attaches to four drives, and there is the option to connect the second cable to enable dual-port functionality. Two large 12V fans provide copious airflow to the chassis, and we can see the dual PSU arrangement.

The 48-pin 500W PSUs are hot swappable, and each has its own AC inlet. The chassis alarm sounds if power is lost to either PSU, or if the chassis fans fail.

55

The drives are plugged into six separate backplanes that are routed to the rear of the chassis. Each backplane has two connections routed directly to the rear of the unit. Power is served from a small PCB at the base of the chassis, and two connectors supply each backplane with power.

The Dual HD MiniSAS Receptacle (SFF-8644) to Dual Int MD MiniSAS Plug (SFF-8643) adaptors are present along the rear of the chassis. These convert the internal receptacles to external. These connectors can be purchased separately from Serial Cables, and we have used them extensively to convert internal RAID controller connections to external cabling.

55

Drive sleds are an often-overlooked component that can have a long-term impact. One of the impressive aspects of the eight-bay JBOD was the smooth drive sled operation, and this held true for the 24-port model as well. Many enclosures are not well aligned with internal backplanes. This creates friction when installing drives, and the friction sometimes requires a bit of force to overcome. The design of the Serial Cables drive sleds provides plenty of airflow through the front of the sled; they are also aligned perfectly, and provide reliable and smooth operation. We tested several types of SSDs and HDDs with the enclosure, and we were impressed with the smooth operation.

Test System and Methodology

55

55

Our approach to storage testing targets long-term performance with a high level of granularity. Many testing methods record peak and average measurements during the test period. These average values give a basic understanding of performance, but fall short in providing the clearest view possible of I/O QoS (Quality of Service).

While under load, all storage solutions deliver variable levels of performance. "Average" results do little to indicate performance variability experienced during actual deployment. The degree of variability is especially pertinent, as many applications can hang or lag as they wait for I/O requests to complete. While this fluctuation is normal, the degree of variability is what separates enterprise storage solutions from typical client-side hardware.

Providing ongoing measurements from our workloads with one-second reporting intervals illustrates product differentiation in relation to I/O QoS. Scatter charts give readers a basic understanding of I/O latency distribution without directly observing numerous graphs. This testing methodology illustrates performance variability, and includes average measurements during the measurement window.

IOPS data that ignores latency is useless. Consistent latency is the goal of every storage solution, and measurements such as Maximum Latency only illuminate the single longest I/O received during testing. This can be misleading, as a single "outlying I/O" can skew the view of an otherwise superb solution. Standard Deviation measurements consider latency distribution, but do not always effectively illustrate I/O distribution with enough granularity to provide a clear picture of system performance. We utilize high-granularity I/O latency charts to illuminate performance during our test runs, and our IOPS to Latency measurements highlight IOPS performance at specific latency points.

Our testing regimen follows SNIA principles to ensure consistent, repeatable testing, and utilizes multithreaded workloads found in typical production environments. The first page of results will provide the "key" to understanding and interpreting our test methodology.

55

We required the utmost performance to test the Serial Cables SA-ENC12G-024A 24-port Direct Attach JBOD, and the HGST SSD800MH provides the best result in steady state of the available 12Gb/s SSDs. The HGST SSD800MH SSDs feature sequential read/write speeds of 1200/750 MB/s, and read/write IOPS of 145,000/100,000 IOPS.

55

We paired the HGST SSDs with the Adaptec ASR-8885 12Gb/s RAID controller, and we conducted our tests at the default controller settings. In our testing, we are searching for any performance variation between two arrays; one array is directly attached via cabling to the RAID controller, and the other array is connected via the Serial Cables enclosure.

Benchmarks - 4k Random Read/Write

4k Random Read/Write

55

We precondition the array of eight HGST SSD800MH SSDs for 9,000 seconds, or two and a half hours, receiving performance reports every second. We plot this data to illustrate the drives' descent into steady state.

This dual-axis chart consists of 18,000 data points, with the IOPS on the left, and the latency on the right. The red dots signify IOPS, and the grey dots are latency measurements during the test. We place latency data in a logarithmic scale to bring it into comparison range. The lines through the data scatter are the average during the test. This type of testing presents standard deviation and maximum/minimum I/O in a visual manner.

Note that the IOPS and latency figures are nearly mirror images of each other. This illustrates high-granularity testing can give our readers a good feel for latency distribution by viewing IOPS at one-second intervals. This should be in mind when viewing our test results below. This downward slope of performance only occurs during the first few hours of use, and we present precondition results only to confirm steady state convergence.

55

Each level tested includes 300 data points (five minutes of one second reports) to illustrate performance variability. The line for each OIO depth represents the average speed reported during the five-minute interval. We include a smaller embedded chart that lists performance at the highest load for easy interpretation.

4k random speed measurements are an important metric when comparing drive performance, as the hardest type of file access for any storage solution to master is small-file random. 4k random performance is a heavily marketed figure, and is one of the most sought-after performance specifications.

In our testing, we are searching for any performance variation between two arrays; one array is directly attached via cabling to the RAID controller, and the other array is connected via the Serial Cables enclosure. If the Serial Cables enclosure provides little to no effect to the native performance of the 12Gb/s SAS SSDs, the results will be somewhat repetitive. This holds true as we start our testing with both arrays reaching 515,000 IOPS, with very little performance variation between direct cabling and the Serial Cables enclosure. We do note some variability at 64 OIO, but this is likely due to internal drive management.

55

55

Our Latency v IOPS charts compare the amount of performance attained from each solution at specific latency measurements. Many applications have specific latency requirements. These charts present relevant metrics in an easy-to-read manner for readers who are familiar with their application requirements.

Both methods of connection, cable and JBOD, exhibit such small latency variation that the lines overlap perfectly in our IOPS v Latency test.

55

Garbage collection routines are more pronounced in heavy write workloads, leading to performance variability.

4k random write workloads are more demanding for the SSDs, and here we can see the enclosure performing admirably with no impact to the base performance of the SSDs.

55

55

The 4k random write results mirror the results we received with random read testing, and the line displaying the JBOD's performance overlaps the DAS connection perfectly.

55

55

Our write percentage testing illustrates the varying performance of each solution with mixed workloads. The 100% column to the right is a pure 4k write workload, and 0% represents a pure 4k read workload.

The JBOD remains impressively consistent, even with mixed workloads compared to a direct-cabled connection. Both latency and IOPS performance are strikingly similar.

Benchmarks - 8k Random Read/Write

8k Random Read/Write

55

55

Many server workloads rely heavily upon 8k performance, and we include this as a standard with each evaluation. Many of our server workloads also test 8k performance with various mixed read/write distributions.

The performance of the HGST 12Gb/s SSD array is impressive, and delivers roughly 500,000 IOPS of performance that is unhindered by the JBOD, which scores a very similar result.

55

55

There is little to no variation in performance and latency during the measurement window.

55

The 8k random write speed averages to 460,000 IOPS, with little variation between the direct cabling and JBOD deployments.

55

55

The JBOD connection results line up neatly with the direct cabling attachment method.

55

55

The mixed results also paint a similar picture, but we do note some variability from 60-80% write mixtures. Both arrays experience the same variability, so this can be chalked up to internal drive management, or RAID controller processes.

Benchmarks - 128k Sequential Read/Write

128k Sequential Read/Write

55

55

128k sequential speed reflects the maximum sequential throughput of the SSD, and is indicative of performance in OLAP, batch processing, streaming, content delivery applications, and backup scenarios. The array blasts out 5.2 GB/s of sequential read performance with both connections at 256 OIO. The very similar performance trace at 64 OIO confirms our theory that some drive, and/or RAID controller processes are in play during the measurement window. Both arrays exhibit the same performance profile at that outstanding I/O count.

55

55

The results fall perfectly in line with each other, even with sequential data.

55

Sequential write performance is important in tasks such as caching, replication, HPC, and database logging. The arrays provide 4.7 GB/s of performance in the sequential write workload.

55

55

Latency and IOPS performance continue to fall within expectations.

55

55

Our mixed sequential workloads are very effective at teasing out areas of differentiation in all manners of storage devices, but there is simply nothing to be seen here. The Serial Cables JBOD continues to provide unfettered performance to the array of 12Gb/s SSDs.

Benchmarks - Database/OLTP and Web Server

Database/OLTP

55

55

This test consists of Database and On-Line Transaction Processing (OLTP) workloads. OLTP is the processing of transactions such as credit cards and high frequency trading in the financial sector. Databases are the bread and butter of many enterprise deployments. These demanding 8k random workloads with a 66 percent read and 33 percent write distribution bring even the best solutions down to earth. Both arrays average 390,000 IOPS, with the JBOD connection actually performing within a slightly tighter performance profile in comparison to the direct-cable connection.

55

55

The slightly lower variability has little impact on the overall performance curve, and we continue to see no discernable difference between the two connection methods.

Web Server

55

55

The Web Server workload is read-only with a wide range of file sizes. Web servers are responsible for generating content users view over the Internet, much like the very page you are reading. The speed of the underlying storage system has a massive impact on the speed and responsiveness of the server hosting the website.

The wide range of small random reads in this test are another good tool for fleshing out differences in products, but this demanding test results in an expected performance profile that varies little between connection methods.

55

55

Benchmarks - Email Server

Email Server

55

55

The email server workload is a demanding 8K test with a 50% read and 50% write distribution. This application is indicative of the performance in heavy random write workloads.

The arrays top out at 395,000 IOPS at 256 OIO.

55

55

Once again, the Serial Cables JBOD provides an unadulterated connection that closely mirrors a direct-cabled connection.

Final Thoughts

55

JBOD testing focuses on ease-of-use, design elements, and whether or not the device is rugged enough for the rigors of deployment. The Serial Cables 24-bay 12Gb/s SAS JBOD features rugged construction and smooth-latching drive sleds that will withstand the wears of time and many drive insertion and removal cycles. This is critical for test environments, as well as for any standard deployment. With no internal logic on our test sample, the chassis is about as simple as it gets, and the focus turns to performance capabilities.

12Gb/s SAS requires receive and transmit equalization, and a JBOD's primary job is to deliver an unadulterated connection between the host and the storage devices. The Serial Cables JBOD employs quality internal components that provide a low-latency connection for an unadulterated shot to the storage device from the motherboard, RAID controller, or HBA.

Our testing environment featured a 12Gb/s SAS Adaptec ASR-8885 connected to an array of HGST SSD800MH SSDs. The Adaptec controller is the fastest controller available with the requisite external connectivity required to interface with the JBOD chassis. The HGST SSDs are the fastest 12Gb/s SAS SSDs on the market, and stacking eight of them in a RAID 0 configuration provided us with the speed to push the boundaries of RAID performance. RAID 0 was chosen to provide the highest throughput possible from our eight SSDs.

With the RAID controller and SSDs in place, the key was measuring any performance differential between arrays connected via the JBOD, and the same array connected directly to the SSDs with standard SAS breakout cables. Our testing focused more on ascertaining the performance impact of utilizing the Serial Cables JBOD for high-performance lab environments. Our testing was actually quite straightforward; we noted no loss in performance by utilizing the JBOD. The JBOD provides a transparent means of connecting storage devices that leverages quality components with a good history of reliability; many of the same design elements are in place on the original eight-bay JBOD.

We monitored the temperature of the HGST SSDs during the rigors of random and sequential testing, and the drives remained very cool within the chassis. The Serial Cables JBOD employs two large fans that provide a copious amount of airflow, but at the same time, they generate quite a bit of noise. The JBOD is destined for rack environments where noise will not be a concern, and airflow is of utmost concern.

Serial Cables also included redundant PSUs in this revision, which are much easier to accommodate in the 19" 2U chassis. This provides a level of redundancy that could see this JBOD thrive in a production environment. The JBOD also supports dual-port connections, which allow for testing either failover or multipath, or use in a datacenter applications. Of course, any deployment into a normal use comes with the caveat that there is no internal expander, but models are available with different expander configurations.

The Serial Cables 24-Bay JBOD performed well in our tests, and delivered the full performance of the drives in the enclosure. The enclosure comes with a one-year warranty, which might restrict its use in some environments. The low cost of the device, especially in comparison to other components in this class, wins the TweakTown Best Value Award.