GIGABYTE MD60-SC0 (Intel C612) Server Motherboard Review

Introduction and Packaging

We are starting to receive GIGABYTE server motherboards that support Haswell-EP E5-2600 v3 processors and DDR4 memory. Today we have our first sample, a storage centric board sporting dual CPUs and a large number of storage options.

Besides the usual performance gains of moving to a Haswell-EP platform, GIGABYTE has added some of its own improvements to its board designs. The improvements GIGABYTE has made include an all IR power design that uses IRPowlRstage IC controllers, OS-CON capacitors for long life, and high-end ferrite core chokes for stable power delivery.

There is also an interesting Easy BIOS update capability that allows you to flash the BIOS without utilities, CPUs installed, or even memory and power. This update can even be done over your IPMI interface, and allows multiple board updates simultaneously with a simple command line.

GIGABYTE has also updated its IPMI web interface to 2.0, which adds even more remote access features. Now, let's check out the new GIGABYTE MD60-SC0 server motherboard.

Packaging

Here we see the front of the retail box for the MD60-SC0. This is just like several other boxes we have seen from GIGABYTE, and the only marking that indicates what motherboard is inside is a white sticker on the side. The box itself has nice, clean graphics that can be used for many different types of motherboards; this helps to lower production costs, which is great.

Just like we saw on the front of the box, here we see the same generic design on the back that we have seen with several GIGABYTE products.

The insides are rather simple; there is foam padding to protect the motherboard on all sides and the motherboard is wrapped in anti-static bag for additional protection.

After removing the motherboard and foam inserts, we found a cardboard insert holding the accessories. Our sample board came with a few extra accessories that you would not normally get with this kit. These are the accessories listed on the GIGABYTE website:

• 1 x Driver CD
• 1 x I/O shield
• 1 x User manual
• 2 x SATA III 6Gb/s cables

Specifications and Layout

Specifications

The specifications for the MD60-SC0 are typical for a server motherboard, and include four PCIe slots, two mini-SAS ports, ten SATA ports, and three high-speed GbE LAN ports for network connections. The MD60-SC0 also includes a dedicated management LAN port.

Here we see a detailed block diagram showing how all of the input/output ports connect to different chips on the motherboard.

Layout

Here we get our first look at the MD60-SC0 server motherboard. The bottom half of the motherboard is dominated by the CPU sockets and RAM slots that stretch clear across the board from side to side. The orientation of the sockets allows the best cooling possible with a fan bar right in front of this area. Here is the full list of connections and headers found on the MD60-SC0:

• 1 x 24-pin ATX main power connector
• 2 x 8-pin ATX 12V power connectors
• 8 x SAS 6Gb/s ports
• 2 x SATA III 6Gb/s ports
• 2 x Mini-SAS cable connectors
• 2 x CPU fan headers
• 5 x System fan headers
• 1 x USB 3.0 header
• 1 x Serial header
• 1 x TPM header
• 1 x Front panel header
• 2 x LAN activity LED headers
• 1 x HDD back plane board header
• 2 x SAS SGPIO headers
• 1 x Chassis intrusion header
• 1 x Management LAN port link LED header
• 1 x Management LAN port active LED header
• 1 x PMBus connector
• 1 x IPMB connector
• 2 x SATA DOM support jumpers
• 1 x Clear CMOS jumper
• 1 x ME update jumper
• 1 x Clearing supervisor password jumper
• 1 x BIOS recovery jumper
• 1 x ME recovery jumper
• 1 x S3 power-on select jumper
• 1 x Intel / LSI software RAID key jumper
• 1 x Force to stop FRB timer jumper

Here we see the PCIe slots, which are:

• 2x PCIe x16 (Gen3 x16 bus) slots
• 2x PCIe x8 (Gen3 x8 bus) slot
• 2x Mini-SAS cable connector which support SATA3 6Gb/s
• 1x Mezzanine card Type T (Gen3 x8 bus) on the left edge
• 2x SATA 3 6Gb/s connector (Red)

Our sample came equipped with a CRA220-8 add-on SAS card, which you can see on the left edge.

Now we are looking at the upper right side of the motherboard. Right above the square heat sink for the 10GbE Intel 82599ES Network Controller, we have a 10GbE QFSP+ LAN port (Intel 82599ES) supporting dual 10GbE SFP+ output via splitter cable. To the left of that, there are two GbE LAN ports (Intel I350-AM2), with one 10/100/1000 management LAN next to that.

Here we are looking at the lower left section of the motherboard. Along the right-hand edge of the motherboard, we find eight SAS 6Gb/s connectors (GIGABYTE extension card required), then the usual power connectors and fan headers along the front edge of the board.

The lower right side of the motherboard has the main 24-pin power connector at the bottom edge on the motherboard.

Now we are looking at the back I/O ports. At the left, we see a PS/2 mouse/keyboard port with two USB 3.0 ports below. The next stack is a serial port above a video port, and after that is the 40 Gbps QSFP LAN port. The next stack has a Gigabit LAN port with two USB 2.0 ports below. The following two stacks have a Gigabit LAN port, and two USB 3.0 ports below. The final stack has a KVM Server Management Mbps LAN Dedicated LAN port.

As an optional accessory, our MD60-SC0 came equipped with a CRA220-8 SAS 6Gb/s add-on card. This card supports a LSI SAS 2308 controller, and is a Mezzanine card Type T (Gen3 x8 bus), which fits into the slots on the left edge of the board. This card provides support for the eight blue 8x SAS 6Gb/s connectors on the left edge of the motherboard.

BIOS and Remote Management

BIOS

The MD60-SC0 BIOS is typical for server motherboards.

We are now looking at the main screen when you enter the BIOS. Here you can see basic information on the motherboard, BIOS version, memory, and the date and time.

The rest of the BIOS screen shots are typical for motherboards of this type, so we will just show what is available.

Remote Management

To get remote access up and running, enter the BIOS and head over to the server management tab. Next, move down to the BMC LAN configuration tab, which will show the IP address used for remote access.

Enter this IP address into your browser, and you will see the login page.

Log-in using:

Username: admin

Password: password

As a best practice, administrative users should change factory default username/password logins before connecting any new server to their network.

The rest of the screens are typical for GIGABYTE remote access, so we will just show the screens here.

Test System Setup

We would like to thank GIGABYTE, Crucial, SanDisk, Yokogawa, Thermaltake, Noctua, SPEC, Passmark, Primate Labs, and AIDA64 for their support in providing parts for our test system.

The processor we will be using is the Intel Xeon E5-2698 v3, which features 16 cores with hyper-threading.

The Intel C612 chipset will have support provided for up to 18 core processors.

In our tests, we will be using the new Crucial DDR4 memory, which has a speed of 2133 MHz, and a rating of CL15. We will use eight 16GB sticks of these kits, and that will bring us to 128GB of RAM.

We have already taken a look at these memory kits, and you can find our review for those here: Crucial DDR4-2133 DRx4 RDIMM Memory Review - Testing up to 256GB.

Here we can see the timings of the Crucial DDR4 memory that we will be using in our tests.

The memory setup page describes how different configurations and types of memory used will effect memory speeds. Our tests will use a full load of memory, and that will drop our memory speed down to 1866 MHz.

CPU Benchmarks

Cinebench R15

The MD60-SC0 has a good score in CINEBENCH R15, which shows very good performance. This is a storage centric motherboard, so it has been tuned for storage applications.

wPrime

wPrime is a leading multi-threaded benchmark for x86 processors that tests your processor performance. This is a great test to use to rate the system speed; it also works as a stress test to see how well the system cooling is performing.

In wPrime, the MD60-SC0 is tuned for storage applications, so fast wPrime numbers are not that important.

Memory & System Benchmarks

AIDA64

AIDA64 memory bandwidth benchmarks (Memory Read, Memory Write, and Memory Copy) measure the maximum achievable memory data transfer bandwidth.

Memory bandwidth for the MD60-SC0 is doing very good with memory writes, but slightly slower in reads and copy instructions.

Linpack

Intel Optimized LINPACK Benchmark is a generalization of the LINPACK 1000 benchmark. It solves a dense (real*8) system of linear equations (Ax=b), measures the amount of time it takes to factor and solve the system, converts that time into a performance rate, and tests the results for accuracy.

LINPACK is a measure of a computer's floating-point rate of execution ability and measured in GFLOPS (Floating-point Operations per Second); ten-billion FLOPS = ten GFLOPS. LINPACK is a very heavy compute application that can take advantage of the new AVX2 instruction. As it puts a very high load on the system, it is also a good stress test program.

In LINPACK, the MD60-SC0 motherboard is showing strong bandwidth compared to other motherboards. This is an important benchmark, and the MD60-SC0 has very good bandwidth numbers.

Even small increases in scores show a big improvement in performance. Certain parts of this test rely on storage, so using setups with SSDs and RAID 0 would increase those scores.

Geekbench - Stream

Geekbench 3 is Primate Labs' cross-platform processor benchmark, with a new scoring system that separates single-core and multi-core performance, and new workloads that simulate real-world scenarios. It also includes STREAM based memory tests which we will include on our reviews.

Here we are looking at the single-core STREAM memory tests. Bandwidth is showing below average results.

Now we are looking at multi-core STREAM tests. The MD60-SC0 is showing strong memory bandwidth results in this test. This points to the MD60-SC0 strength at multi-threaded applications feeding data to large storage arrays.

UnixBench and SPEC CPU2006v1.2

UnixBench 5.1.3

UnixBench has been around for a long time now, and is a good general purpose bench to test on Linux based systems. This is a system benchmark, and it shows the performance of single-threaded and multi-threaded tasks.

Synthetic benchmarks only show part of the performance of a motherboard. When using tests that are more complex, we will start to see a different trend in the scores. UnixBench shows average results for the MD60-SC0 and E5-2699 v3s.

SPEC CPU2006 v1.2

SPEC CPU2006v1.2 measures compute intensive performance across the system using realistic benchmarks to rate real performance.

In our testing with SPEC CPU2006 we use the basic commands to run these tests.

" Runspec --tune=base --config=tweaktown.cfg ," then " int ," or " fp ."

To do multi-threaded, we add in " --rate=72."

When SPEC CPU first came out, these tests could take up to a week to run, but as computers become faster, our tests now take up to four days for a full run, and even less on some systems.

The user can do many things to effect the results of CPU2006 runs, including compiler optimizations, add-ons like Smartheap, and different commands used to start the tests.

This benchmark has many different commands to use depending on what the user is looking for. For our tests, we used basic commands that run a full test with a base tune.

Here you can see the SPEC scores after full runs for Integer (int) and Floating Point (fp) tests.

Single-core runs show how fast (speed) a CPU can perform a given task. In the multi-core runs, we set SPEC CPU2006v1.2 to use all threads to measure the throughput of the system.

The additional cores/threads of this system have a huge impact on performance in these tests and really show the amount of horsepower that a single socket motherboard has.

Single-threaded results are still very important, but when you need many single-threaded apps to run; moving to a CPU with more cores is the way to go. The MD60-SC0 and Dual E5-2699 v3s start to shine in multi-threaded interger workloads. CPU2006 is showing strong multi-core results on the MD60-SC0. Let's take a look at the individual test result scores.

Looking at the results of single-threaded integer runs, we can get an idea of speed at which the E5-2699 v3s can crunch through the different integer tests. Not all CPUs are equal here, and ones that have a higher speed will perform these tests faster. Naturally, using an overclocked system or CPUs with a higher stock speed will generate higher results.

Now we run the test using all 36 cores/72 threads on the E5-2699 v3 processors to measure the throughput of the system. In this test, more cores/threads will have a greater effect on the outcome.

Just like the integer tests, we now run the floating-point tests in single-threaded (speed) mode.

Here we see the results of the multi-core floating-point run that uses all 36 cores/72 threads on the E5-2699 v3 processors. Like the multi-threaded integer test, more cores/threads will have a greater impact on the test.

Just like we saw in the integer multi-threaded tests, the MD60-SC0 and E5-2699 v3s really take off here.

Power Consumption & Final Thoughts

Power Consumption

We have upgraded our power testing equipment, and now use a Yokogawa WT310 power meter for testing. The Yokogawa WT310 feeds its data through a USB cable to another machine where we can capture the test results.

To test total system power use, we used AIDA64 Stability test to load the CPU, and then recorded the results. We also now add in the power use for a server from off state, to hitting the power button to turn it on, and take it all the way to the desktop. This gives us data on power consumption during the boot up process.

The MD60-SC0 uses ~100 watts at idle on the desktop, and it peaks at ~545 watts under full load. The power use is slightly higher because of the added CRA220-8 SAS controller card.

With the MD60-SC0, we see peak power use of only ~350 watts during the boot up process. The system settles down to ~80 watts after the boot up is completed.

Final Thoughts

This our first look at the dual socket 2011-E server motherboards offered by GIGABYTE, and we were very happy with the results the MD60-SC0 provided.

The MD60-SC0 will find its way into storage servers, and can support a large number of different storage options and setups with just the base motherboard. Adding the CRA220-8 allows even more storage capacity with high-performance 8x SAS ports.

Having a lot of storage capability is great, but you also need to move the data to and from the server with fast pipes, and the MD60-SC0 has onboard capabilities to handle your needs with two GbE LAN ports and a 10 GbE QFSP+ LAN for high-speed access needs. This works out great for virtualized systems, is capable of load balancing, and handles low latency requirements.

The MD60-SC0 we received was very simple to get up and running, the BIOS is simple to navigate, and for the most part, we had no issues. The controller menus are very typical of any system that uses these, and you should feel right at home.

Performance of the MD60-SC0 is very much like other storage/VM related motherboards we have tested in the lab, so we had no real surprises here. Overall, we are pleased with the motherboard, and it performed to our expectations. Our multi-core tests came out very good. This motherboard can take on heavy VM tasks without issues, and coupled with its high-speed network options, this allows the MD60-SC0 to be useful without any additional network cards. There are several more PCIe slots available for network options if that is required.

Packaging for the MD60-SC0 is very much like the MW50-SV0 we reviewed earlier, and we find GIGABYTE serves up a retail package aimed at lowering costs.

We liked the new IPMI 2.0 features, and we had no issues getting this setup over iKVM. The new BIOS flash abilities should make managing these features very easy, and will not require you to have extra CPUs laying around to keep the boards updated.