A couple of weeks ago my curiosity turned a light working weekend into a full on charge in exploring the ins and outs of NAS server file systems. While testing the Crucial RealSSD C300, we moved away from the ASUS P6T7 WS SuperComputer motherboard that was used to test storage released products and started using the GIGABYTE X58A-UD7. This left me with a free system at a key time to start taking a deep dive into file systems on NAS servers. At first it started out as a harmless test with a single drive on a different file system to see what would happen to performance. Forty-eight hours later has evolved into a nonstop Bawls fueled obsession to see how much improvement we could find just by setting our NAS up differently.
NAS server testing takes time. Formatting a large array in RAID 5 or 6 can take up to 24 hours. This is a really long lead time just to get in an hour long test with Intel's NAS Performance Test. Needless to say, this editorial took several hours to complete and including the information in a standard review would make the review of the Thecus N7700 Pro almost unbearable for the reader.
Today we are going to look at several performance tests in three file formats found on many Thecus products. The default setting for Thecus is the EXT3 file system; the same is true on QNAP NAS servers. Thecus also includes the option to use ZFS and XFT while QNAP only offers EXT4 as an alternative. Let's take a look at the finer details and get to exploring performance on the new Thecus N7700 Pro.
To start things off, this article is not designed to be a full review of the Thecus N7700 Pro; that article will come in a few days. Above we can see what we are working with in the Thecus N7700 Pro. As you can see, the N7700 Pro is a very high end unit that has many hardware features not found in entry level NAS servers.
Among the stand outs are the Core2 Duo processor, 4GB of RAM, 7 HDDS, support for 9K Jumbo Frames and the newest feature to hit NAS servers in the 1K USD price range, 10GbE ability with an add-in PCIe card. We did not test with the 10 GbE option since we do not have a 16GbE switch or a host card for our Storage Product Testing Workstation.
Up to this point we have only tested NAS servers with their default file system, EXT3. EXT3 has been a fine file system, but I found that it is possible to gain more performance by simply changing the file system to one of the other formats. Which one was best for performance? You will have to keep reading to find out.
Below are some brief descriptions of each file system found on Thecus hardware. The EXT4 option is not on Thecus products, but is the optional file system on QNAP hardware, Thecus' closest competitor. This information is given for future use as we will be linking back to this article while reviewing hardware at a later date.
EXT3 - The third extended file system is journaled file system that is commonly used by the Linux kernel and in many cases the default file system. The main benefit over EXT2 is journaling which improves reliability and eliminates the need to check the file system after an unclean shutdown.
EXT4 - A revised file system based on EXT3 that removes the 64-bit limits and also adds other performance improvements over EXT3.
XFS - A journaling file system designed for high performance that is proficient at handling large files and offering improved smooth data transfers.
ZFS - A combined file system and logical volume manager that includes support for high storage capacities, volume management, snapshots and copy-on-write clones with continuous checking and automatic repair.
If you would like to dive into hours of reading about each file system, their origins and fine details, you can start at Wikipedia and finish a few days later at your local library. Today we are more interested in seeing how these file systems perform in our real world usage tests via Intel's NAS Performance Toolkit.
Test System Setup
Motherboard: ASUS P6T7 WS SuperComputer (Supplied by ASUS)
Processors: Intel 975 EE (Supplied by AVADirect)
Memory: Corsair Dominator 1600 MHz Triple Channel Kit
Graphics Card: Leadtek Quadro FX1700 (Supplied by Leadtek)
Enclosure: Lian Li V2000
Cooling: Noctua NH-U12P SE2 (Supplied by Noctua)
SAS Controller: LSI MegaRAID SAS 9260-8i (Supplied by LSI)
Operating System: Microsoft Windows Vista Ultimate X64
Test HDDs: Seagate ES.2, 1TB (Supplied by Seagate)
The Intel NAS Performance Toolkit (NASPT) is a file system exerciser and analysis tool designed to enable direct measurement of home network attached storage (NAS) performance. Designed to emulate the behavior of an actual application, NASPT uses a set of real world workload traces gathered from typical digital home applications. Traces of high definition video playback and recording, office productivity applications, video rendering/content creation and more provide a broad range of different application behaviors.
With version 1.71 we are able to test with Windows 7 64-bit, but to keep the results accurate you must limit your system RAM to 2GB or less. With only a single 2GB stick of memory installed you can test with Windows 7 64-bit. Also, a new test was added, Office Productivity, but several tests were removed like 3 Source HD Video Playback and Backup / Restore.
Benchmarks - HD Playback
HD Video Play - 720p HD stream from Windows Media Player* 256kB reads
2HD Video Play - 2x playback
4HD Video Play - 4x playback
Right from the start we see the XFS file system really taking off and leaving EXT3 and ZFS way behind in transfer speeds. In some cases XFS is able to double the speed of the other two file systems. The transfer speed advantage is present from JBOD and goes all the way to the 7 drive, RAID 6 array.
Benchmarks - HD Record
HD Video Record - 720p HD stream, 256kB writes
HD Video Play & Record - 1 playback, 1 record simultaneously
Once again we see the XFS file system step away from the other two file systems, but in these tests the EXT3 file system is not that far behind. It is really interesting to see just how far behind the ZFS system is.
Benchmarks - Content
Photo Album - All reads - wide distribution of sizes
Office Productivity -
Content Creation - 95% writes; 1k, 4k & little reads; Writes up to 64kB
The Content Tests are far the most menacing and put all NAS servers to the test with their small file size transfers. Here we see nearly identical performance numbers for both EXT3 and XFS and once again the ZFS is lagging behind the other file formats.
The newest test for the NASPT is also found here and it is the Office Productivity test. In this test we also see that EXT3 and XFS performance is nearly identical.
Benchmarks - Copy
Directory Copy From NAS - 64kB reads
Directory Copy To NAS - Predominantly 64kB writes, wide scattering under 16kB
File Copy From NAS - 4GB file copy, 64kB reads
File Copy To NAS - 64kB writes
The copy tests once again show higher performance with the XFS file system and its ability to power through all of the tasks with faster transfer rates. The difference between XFS and ZFS is quite staggering in many of the tests with XFS performing 3x as fast in many cases.
What started out as just playing around in the lab turned into a full on assault to find the fastest file system for use on Thecus NAS products. I have already started performing the same type of testing on a QNAP NAS that uses EXT3 and EXT4, but the results are not as significant as the XFS file system is on the Thecus hardware.
What we have discovered could really be a game changer for people shopping for a NAS server and looking to get the absolute highest performance in regards to transfer rates out of their long term storage systems. No one likes to wait for file transfers to complete and with XFS it appears that you can achieve very high data rates with fewer drives. XFS also seems to take the performance hit out of RAID 5 and RAID 6 arrays.
Many people that I know who purchase or are looking to purchase a NAS are looking to start out with three or four drives in RAID 5 and look to build upon the array as their storage needs increase. A three drive RAID 5 array has historically meant a compromise in data transfer performance, but with XFS it appears that is no longer the case in many tests. There are always going to be certain scenarios that are slower than most like the working with Windows Photo Album or dealing with many small files and in those cases we didn't see much of an improvement between the default EXT3 and XFS file system.
It is troubling to see that what appears on paper (errr, monitor?) as the most advanced file system, ZFS , is also the slowest performing by a large margin. From reading online and other resources it appears that ZFS has several strong features that would make it ideal for mission critical installations where your data would need the absolute best protection, but for many users a standard RAID 5 array will do the trick.
I applaud Thecus for providing stuck wide spread options when it comes to the file systems offered with their NAS products. As I stated earlier already, the QNAP systems I have looked at and run here only offer two, EXT3 and EXT4. As we have seen, the two EXTs are not going to provide you with as high of performance, especially when using only a few drives.
In the reviews of future NAS products we are going to have to put more of an emphasis on the types of file systems offered. It is quite amazing to see just how much of an impact this easy to setup option has on performance.
PRICING: You can find products similar to this one for sale below.
United States: Find other tech and computer products like this over at Amazon.com
United Kingdom: Find other tech and computer products like this over at Amazon.co.uk
Australia: Find other tech and computer products like this over at Amazon.com.au
Canada: Find other tech and computer products like this over at Amazon.ca
Deutschland: Finde andere Technik- und Computerprodukte wie dieses auf Amazon.de