Test System and Methodology
We designed our approach to storage testing to target long-term performance with a high level of granularity. Many testing methods record peak and average measurements during the test period. These average values give a basic understanding of performance, but fall short in providing the clearest view possible of I/O Quality of Service (QoS).
'Average' results do little to indicate performance variability experienced during actual deployment. The degree of variability is especially pertinent as many applications can hang or lag as they wait for I/O requests to complete. This testing methodology illustrates performance variability, and includes average measurements, during the measurement window.
While under load, all storage solutions deliver variable levels of performance. While this fluctuation is normal, the degree of variability is what separates enterprise storage solutions from typical client-side hardware. Providing ongoing measurements from our workloads with one-second reporting intervals illustrates product differentiation in relation to I/O QoS. Scatter charts give readers a basic understanding of I/O latency distribution without directly observing numerous graphs.
Consistent latency is the goal of every storage solution, and measurements such as Maximum Latency only illuminate the single longest I/O received during testing. This can be misleading as a single 'outlying I/O' can skew the view of an otherwise superb solution. Standard Deviation measurements consider latency distribution, but do not always effectively illustrate I/O distribution with enough granularity to provide a clear picture of system performance. We utilize high-granularity I/O latency charts to illuminate performance during our test runs.
Our testing regimen follows key SNIA principles to ensure consistent, repeatable testing. LSI SandForce's inline compression engines give performance a boost with compressible data. Many workloads, including OLTP and database workloads, are highly compressible. Other workloads have varying levels of compressibility. In the interest of representing the broadest swath of use-cases, we test with 50 percent compressible data. The first page of results will provide the 'key' to understanding and interpreting our new test methodology.
The Comay BladeDrive E28 comes with a comprehensive software suit to monitor and manage the SSD. Users can monitor endurance and temperature, and can even secure erase the drive. The GUI also offers basic benchmarking capabilities and partition management. The GUI is not as polished as other solutions but is clearly a work in progress. This inaugural management utility for their new series of PCIe SSDs is a good start.