RAID controllers are a lot of things to different people. Some like me, a power user, is able to tune the RAID block size for general use computing, balancing the small file IOPS with large sequential file transfer speeds. Enterprise users running database tasks on their controller can tune the block size to achieve high IOPS when needed. Having the ability to change RAID block sizes is the key to making a RAID controller to fit every occasion, including users looking to stream large files like movies and music.
For years I've talked about RAID tuning and even gave some examples on a small scale. The LSI 9265-8i that we are looking at today takes the ability to optimize your RAID arrays to new levels and does so with a very easy to navigate user interface. The base controller with its dual-core ROC processor and 1GB DRAM cache has become the industry standard, but LSI didn't stop there. Add on packages like LSI's FastPath and CacheCade further enhance usability based on user need.
When our 9265-8i sample first arrived we were in the process of building a super computer based on the EVGA SR-2 dual Xeon enthusiast's board. At the time LSI was having issues with a handful of enthusiast class motherboards with the SR-2 being one of them. Since that time LSI and the motherboard makers have found and corrected several if not all of the issues. There was a time though when enthusiasts were pounding their heads against the wall trying to make everything mesh as it should.
Today we'll be looking at the LSI 9265-8i MegaRAID SAS in several RAID configurations and with different block sizes. Let's dive in and see the specifications.
Specifications, Pricing and Availability
The 9265-8i MegaRAID is from LSI's Value Line of storage controllers but you wouldn't know it from the specification list. The heart of the system is a dual-core 800MHz RAID-on-Chip (ROC) that's paired with an embedded 1GB DDR2 cache. This is LSI's second generation 6Gb/s ROC, we reviewed the first generation last year on the 9260-8i.
Connectivity comes via two SFF-8087 mini-SAS ports, each with 6Gb/s throughput. There are several different configurations for SAS. You could run each channel directly to a SATA or SAS drive or with expanders the 9265-8i can connect up to 128 drives. On the host end the 9265-8i uses a PCIe 2.0 8x lane and this will be your limiting factor when using a very large array or a group of high speed SSDs.
The MegaRAID series of products cover all of the major RAID array types. RAID 5 and RAID 6 are the most popular and allow for one (RAID 5) or two (RAID 6) disks to fail before you lose data. RAID 0, the old enthusiast's favorite is also included, but with the advent of high speed SSDs most stay away from RAID 0 since it doesn't offer any redundant features or significant performance increase over RAID 5. The ability to run RAID spans 10, 50 and 60 are also available.
Current pricing on the 9265-8i is about 40% less at the time of writing than what we found on release in 2011. We managed to track down a seller on Google's Product Search with stock available for $620 shipped in the US. That covers the basic card, SFF-8087 to SATA cables (2), the manuals and a three year warranty. Add-ons include a battery backup unit, MegaRAID FastPath, SafeStore, CacheCade and recovery software. A fully outfitted 9265-8i with all of the goodies can add another $870 if purchased through LSI directly. For most users, Fast Path and CacheCade are all that is needed. We managed to find a bundle package with both of these add-on packs for 249.99.
Over the years we've played with quite a few RAID controllers and built many arrays. I have to admit that LSI's MegaRAID Management Suite is by far the most attractive and easiest to work with. The software can be run out of band as well making it possible to control your server from another system, a big plus when running a datacenter or cluster.
Just because the MegaRAID software is the easiest to work with doesn't mean it doesn't have a learning curve. Depending on what you highlight, controller, backplane or individual drive, the Go To menu will give you different options. The physical and logical tabs will also have you twisted around if trying to make a quick correction without putting a little thought into what you want to accomplish.
The additional software features that are available add to the overall value as well. I'm not going to rewrite LSI's marketing, here are the quotes.
1. LSI MegaRAID FastPath
LSI MegaRAID FastPath optimization software provides a high performance I/O accelerator, designed to dramatically boost transactional application throughput of multiple SSDs connected to a 6Gb/s SATA+SAS MegaRAID controller. FastPath supports full optimization of SSD Virtual Disk groups to deliver a 3X improvement in read and write I/Os per second (IOPS) compared to the previous generation product.
2. LSI MegaRAID CacheCade
LSI MegaRAID CacheCade tiered cache allows users to leverage SSDs in front of hard disk drives (HDDs) to create up to 512GB of controller cache. Using SSDs as controller cache allows for very large data sets to be present in cache to deliver up to a 50X performance improvement in read-intensive applications, such as file, Web, OLTP and database server. The solution is designed to provide a dramatic performance upgrade while only requiring a small investment in SSD technology.
3. LSI MegaRAID Recovery
LSI MegaRAID Recovery enhances file and boot-block/volume protection, empowering IT administrators with data retrieval and upgrade risk-management. Users can capture source volume data at designated Points-in-Time (PiT) and "restore " or "rollback" should accidental or malicious data deletion or infection occur. While traditional periodic off-hour backups continue to be the primary method for preventing complete data loss, MegaRAID Recovery meets a growing need for more frequent restore points and reduced recovery times. By supplementing traditional backup methods with MegaRAID Recovery, IT organizations can avoid unexpected downtime and greatly enhance their level of business continuity, data protection and restore capabilities.
4. LSI SafeStore Encryption Services
Data security also has been enhanced utilizing LSI SafeStore Encryption Services. This feature, together with self-encrypting drives (SEDs), secures a drive's data from unauthorized access or modification resulting from theft, loss or repurposing of drives. Local key management in the controller is designed to protect data from security breaches with minimal cost and complexity. Another benefit of SafeStore with SED drives is the ability to instantly and securely make data on disk drives unreadable, significantly reducing the time and cost associated with secure drive erasure.
To date I've used Fast Path and CacheCade. Today we'll be using Fast Path with our array for Crucial C300 256GB SSDs. LSI includes a feature called SSD Guard (included with the bare drive) and we will use that as well.
LSI MegaRAID 9265-8i
We're not going to spend a whole lot of time today going on the packaging. Most, if not all, 9265-8i's will be sold over the net.
LSI has information all over the box - front, back and sides.
Everything is packaged well inside with the controller separate from the accessory package.
Speaking of the accessory package, you get two cables, a low profile adapter bracket, several manuals and finally a software disk with drivers and more manuals.
Here we get our first look at the LSI 9265-8i controller. This is a passive cooled controller so you'll want some air flow going to the card. Our sample is currently sitting in an open case with a little breeze coming from a fan mounted around 8 inches away, but the smaller your enclosure, the more air you'll need.
The two SFF-8087 ports, each with four 6Gb/s lanes are at the front of the card. This is a little different than most of the controllers we see with the cables coming off the back. The design let's you use a shorter depth case, but you will be somewhat tight for space when using a 2U enclosure with a vertical PCIe stack.
LSI covered the card with surface mount components; they are just about everywhere with very little surface left open.
A daughter board holds the cache on the back of the card.
Two brackets are included. We used the full height version, but 2U users can easily switch to the half-height bracket.
Benchmarks - Test System Setup and ATTO Baseline Performance
We would like to thank the following companies for supplying and supporting us with our test system hardware and equipment: AVADirect, GIGABYTE, Cooler Master, LSI, Corsair. and Noctua.
You can read more about TweakTown's Storage Product Testing Workstation and the procedures followed to test products in this article.
As we mentioned previously we'll display several arrays today with the LSI MegaRAID 9265-8i. RAID 5 and RAID 6 will use a 64k size, the default when building an array. RAID 0 performance will be shown in 8kb, 64kb, 512kb and 1MB.
I've wanted to write a review in this format for years and this article took a very long time to finally finish. What you see today is only a small part of the tests run. We also tested with Seagate 15,000 RPM SAS drives, Corsair Performance 3's and a few oddball arrays with and without Fast Path. We'll also show the Fast Path results as well.
For the review we are using eight Crucial C300 256GB drives with and without Fast Path enabled. Fast Path is the additional software that increases the IOPS performance of MegaRAID controllers. In RAID 0, 5 and 6 you double IOPS performance when used with drives that are capable of delivering high IOPS performance. Achieving 2x performance under real-world conditions would be very difficult, unlike the targeted tests.
ATTO Baseline Performance
Version and / or Patch Used: 2.34
ATTO is used by many disk manufacturers to determine the read and write speeds that will be presented to customers.
In this ATTO run we see an eight drive (8D) RAID 0 (R0) array with a 1MB build. The read performance reaches 3500 MB/s while write performance is over 2770 MB/s. The 4k write speed is over 321 MB/s.
Benchmarks - HD Tune Pro
HD Tune Pro
Version and / or Patch Used: 4.00
Developer Homepage: http://www.efdsoftware.com
Product Homepage: http://www.hdtune.com
HD Tune is a Hard Disk utility which has the following functions:
Benchmark: measures the performance
Info: shows detailed information
Health: checks the health status by using SMART
Error Scan: scans the surface for errors
HD Tune Pro gives us accurate read, write and access time results and for the last couple of years has been gaining popularity amongst reviewers. It is now considered a must have application for storage device testing.
As you can see we are analyzing from a pretty deep well today. We'll start out with the obvious, Fast Path enabled gives us a nice performance increase when comparing equal block sizes. This is a sequential read test so the larger stripe sizes performed better than the smaller sizes. Last but not least, RAID 5 and RAID 6 didn't have a very large performance penalty at all with the controller running eight drives.
In the sequential write test Fast Path once again gave us a nice boost. In the larger stripe sizes the difference was sometimes over 300 MB/s (average write). The smaller strip sizes couldn't keep up with the larger stripe sizes; observe the difference between 64kb vs. 512kb and 1MB.
Benchmarks - AIDA64 Random Access Time
AIDA64 Random Access Time
Version and / or Patch Used: 1.60
Developer Homepage: http://www.aida64.com
Product Homepage: http://www.aida64.com
AIDA64 offers several different benchmarks for testing and optimizing your system or network. The Random Access test is one of very few if not only that will measure hard drives random access times in hundredths of milliseconds as oppose to tens of milliseconds.
Drives with only one or two tests displayed in the write test mean that they have failed the test and their Maximum and possibly their Average Scores were very high after the cached fills. This usually happens only with controllers manufactured by JMicron and Toshiba.
In the read access times I came away a little surprised. Here I figured the smaller block sizes would dominate the charts, but that wasn't the case. Fast Path on the other hand did an amazing job keeping latency to around .12ms in all of the read tests when the feature was enabled.
Once again I expect to see the 8kb stripe size do much better with the write latency, but that wasn't the case. RAID 5 and RAID 6 saw an increase in write latency, but that is to be expected. Fast Path kept the redundancy levels latency in check for the most part.
Benchmarks - CrystalDiskMark
Version and / or Patch Used: 3.0 Technical Preview
Developer Homepage: http://crystalmark.info
Product Homepage: http://crystalmark.info/software/CrystalDiskMark/index-e.html
Download here: http://crystaldew.info/category/software/crystaldiskmark
CrystalDiskMark is a disk benchmark software that allows us to benchmark 4K and 4K queue depths with accuracy.
* Sequential reads/writes
* Random 4KB/512KB reads/writes
* Text copy
* Change dialog design
* internationalization (i18n)
Note: Crystal Disk Mark 3.0 Technical Preview was used for these tests since it offers the ability to measure native command queuing at 4 and 32.
In CDM we're looking at 4K and native command queuing performance. LSI's default stripe size is 64kb and it looks like they choose that size for a reason.
At this point we can call the 8kb stripe size worthless, the drives can't ramp up the speed in that small of a stripe. Fast Path gives us a small advantage in these tests, but much less than I expected.
Benchmarks - PCMark Vantage Hard Disk Tests
PCMark Vantage - Hard Disk Tests
Version and / or Patch Used: 1.0.0
Developer Homepage: http://www.futuremark.com
Product Homepage: http://www.futuremark.com/benchmarks/pcmark-vantage/
PCMark Vantage is the first objective hardware performance benchmark for PCs running 32 and 64 bit versions of Microsoft Windows Vista. PCMark Vantage is perfectly suited for benchmarking any type of Microsoft Windows Vista PC from multimedia home entertainment systems and laptops to dedicated workstations and high-end gaming rigs. Regardless of whether the benchmarker is an artist or an IT Professional, PCMark Vantage shows the user where their system soars or falls flat, and how to get the most performance possible out of their hardware. PCMark Vantage is easy enough for even the most casual enthusiast to use yet supports in-depth, professional industry grade testing.
FutureMark has developed a good set of hard disk tests for their PCMark Vantage Suite. Windows users can count on Vantage to show them how a drive will perform in normal day to day usage scenarios. For most users these are the tests that matter since many of the old hat ways to measure performance have become ineffective to measure true Windows performance.
HDD1 - Windows Defender
HDD2 - Gaming
HDD3 - Windows Photo Gallery
HDD4 - Vista Startup
HDD5 - Windows Movie Maker
HDD6 - Windows Media Center
HDD7 - Windows Media Player
HDD8 - Application Loading
The LSI MegaRAID 9265-8i isn't designed for general desktop use, but several of us use them just for that purpose. In the beginning of this article I mentioned that RAID 5 and RAID 6 don't have a performance penalty with SSDs like they did spinners from years past. You can see that effect in this test when comparing all of the 64kb results with and without Fast Path. Fast Path does increase performance by a good margin in these tests. Also, 64kb appears to be the sweet spot for the stripe sizes, the larger blocks trail behind in many of the daily use tests where a mixed load is normal.
Benchmarks - AS SSD
AS SSD Benchmark
Version and / or Patch Used: 1.2.3577.40358
Developer Homepage: Alex Intelligent Software
Product Homepage: Alex Intelligent Software
Download here: http://www.alex-is.de/PHP/fusion/downloads.php?cat_id=4&download_id=9
AS determines the performance of Solid State Drives (SSD). The tool contains four synthetic as well as three practice tests. The synthetic tests are to determine the sequential and random read and write performance of the SSD. These tests are carried out without the use of the operating system caches.
In all synthetic tests the test file size is 1GB. AS can also determine the access time of the SSD, the access of which the drive is determined to read through the entire capacity of the SSD (Full Stroke). The write access test is only to be met with a 1 GB big test file. At the end of the tests three values for the read and write as well as the overall performance will be issued. In addition to the calculated values which are shown in MB/s, they are also represented in IO per seconds (IOPS).
Note: AS SSD is a great benchmark for many tests, but since Crystal Disk Mark covers a broader range of 4K tests and HD Tune Pro covering sequential speeds, we will only use the Copy Benchmark from AS SSD.
- Copy Benchmark
When it comes to transferring ISO files from one area to the drive to another Fast Path actually slows the transfer down in the large stripe sizes. The 64kb stripe size without Fast path is the fastest on the chart.
When you get into the mix between small and large files in the Program Test Fast Path significantly increases performance.
Benchmarks - Passmark
Passmark Advanced Multi-User Tests
Version and / or Patch Used: 6.1
Developer Homepage: http://www.passmark.com
Test Homepage: http://www.passmark.com
Many users complain that I/O Meter is too complicated of a benchmark to replicate results so my quest to find an alternative was started. Passmark has added several multi-user tests that measure a hard drives ability to operate in a multi-user environment.
The tests use different settings to mimic basic multi-user operations as they would play out on your server. Variances is read / write percentage as well as random / sequential reads are common in certain applications, Web Servers read nearly 100% of the time while Database Servers write a small amount of data.
The Workstation test is the only single user environment and will be similar to how you use your system at home.
In the server tests we're mainly looking at the Database Tests (light blue). Databases are extremely hard on HDDs, SSDs and RAID arrays. The small reads and writes are in such small sizes it's difficult to increase performance. Here, just a little performance improvement is a significant upgrade.
Once again we see Fast Path at a 64kb stripe size outperforming the other configurations. Even the RAID 5 and RAID 6 arrays with Fast Path and a 64kb stripe size do very well compared to the other stripe sizes.
If I were to ever tackle a project like this again I would make the 32kb block size my lowest limit and forgo 8kb all together. The 8kb stripe size didn't do anything for us other than disappoint and somehow manage to kill three of the drives. Luckily we started out with twenty six C300s and experiments like this are why we have them.
Three years ago we started testing 6Gb/s SATA and SAS controllers. Of the first generation the LSI 9260-8i was the most impressive. The controller was amazingly fast, even without Fast Path. Once we added Fast Path performance increased significantly. The 9260-8i in now in my daily use workstation running four LSI / SandForce controlled OCZ Vertex 3 MaxIOPS drives. The performance is unbelievable, I always get the jet in BF3 and there is nothing you can do about it! The 9260-8i is no longer the fastest RAID controller on the market. It's the second fastest.
The LSI MegaRAID 9265-8i is LSI's second generation 6Gb/s controller and is a even faster than the MegaRAID 9260 generation. LSI made several changes between the two models. The second generation doubles cache capacity (512MB to 1GB) and nearly doubled the speed (800MHz DDR-II to 1333MHz DDR-III). The 9265-8i also uses a new dual-core ROC running at 800 MHz which allows RAID 5 and RAID 6 to perform at higher levels. The new ROC is so fast that RAID 5 and RAID 6 perform faster than RAID 0 on the previous generation controller.
At the time of writing the LSI MegaRAID 9265-8i was available for just a hair over $600. When our sample arrived the price was closer to $1000, so the 40% reduction is welcome especially in this economy. LSI's competitors also have similar products at this price point, but not as fast as the 9265-8i.
Getting the most performance out of your LSI MegaRAID 9265-8i requires a Fast Path key which adds to the overall cost. Another option for price conscious users is to use a single high speed solid state drive with an array of mechanical drives in CacheCade. This configuration keeps your hot data on the SSD for SSD like reads. CacheCade works very well in web servers where the same small files are read more than other data.
LSI has shown us the future of enterprise storage technology today. The fusion between raw disk speed and advanced software to further increase performance will continue on. Next stop, 12Gb/s!