ASRock EP2C612D16FM (Intel C612 and DDR4) Server Motherboard Review

Introduction and Packaging

Today we are going to look at the ASRock EP2C612D16FM dual-socket motherboard that is ready to run the new Haswell-EP processors and DDR4.

We have in the lab the newly released Haswell-EP processors and DDR4 ready to roll with this motherboard. ASRock Rack is releasing its new EP2C612D16FM motherboard, which is geared to handle a large amount of storage devices on a small form factor, dual-socket platform.

The EP2C612D16FM is designed to fit into a case that has the main storage bays up front, and includes a series of four edge connectors that make connecting to the backplane simple and clean. We have not yet seen what these new systems will look like fully assembled, but this design does look promising, and should make a nice compact system that allows more drives to be installed in front bays.

Popular Now: Grand Theft Auto 6 release date confirmed by Take-Two

Here we have the block diagram of how this motherboard is laid out. The Haswell-EP platform can handle 40 PCIe lanes, and as you can see on this motherboard, these are used to connect to the four edge connectors at the front of the motherboard. The Mezzanine expansion slot is geared for additional network expansion. The C612 chipset feeds plenty of USB and SATA ports to support a large array of storage devices. Let's get started by looking at the ASRock EP2C612D16FM motherboard.

Packaging

Here we can see the front of the retail box. The EP2C612D16FM motherboard comes with a simple retail box that can be used for many different types of motherboards. We think the white box under the word "Workstation" will eventually have a product description, or a label of some kind placed on the outer area.

The back of the retail box is also of a generic type.

We get our first look at the contents of the retail box for the EP2C612D16FM motherboard. The kit provides all the cables needed to connect this motherboard to your devices.

Opening the retail box, we can see the accessories on the top. The parts list for the retail box is as follows:

• ASRock Rack EP2C612D16FM / EP2C612D16FM-N Motherboard (ATX Form Factor: 12.0-in x 10.5-in, 30.5 cm x 26.7 cm)
• Support CD
• User Manual
• 4 x SATA3 Cables (50cm)
• 2 x MiniSAS Cables(6G) (60cm)
• 1 x I/O Shield

Specifications, Layout, and Installation

Specifications

The main supporting features of this motherboard are its large number of Mini-SAS and network ports, which provide storage and RAID options right out of the box, and also plenty of expansion options. The LAN connections also supply basic network connectivity, and provide the option to install faster network cards into the Mezzanine expansion slot. The board also comes equipped with IPMI/iKVM as standard features.

Layout

Here we get our first look at the ASRock EP2C612D16FM motherboard. With a size of 12" x 10.5", this is a very small server motherboard, but there are a massive amount of features packed into this tiny area. The ASRock EP2C612D16FM motherboard supports the Intel Xeon E5-2600 v3 processors and DDR4 memory.

Here we get a look at the main storage connections for the ASRock EP2C612D16FM motherboard. We can see the two Mini-SAS connectors next to the four white SATA III ports. Up at the top we have a Type A USB 3.0 connection, with a blue USB 2.0 header below. Next to those there is a SATA SGPIO header, and a USB 3.0 header. Following down the left side, there is a TPM header, a speaker header, an auxiliary panel header, and system panel headers. The CPU2 power connection is right below the square chipset heat sink.

Moving on to the rear right side of the motherboard, we see the white Mezzanine connector with chassis ID jumper blocks. Following along we find a PMBUS mode jumper block, PSU SMBus, and BMC SMBus header. In-between those there is CPU1 power connector, and the ATX power connector to the right.

On the front left of the motherboard we have a CPU PECI jumper, and two fan connectors. The two big connectors are con ports; these are used to connect the system to a back plane.

On the front right side of the motherboard, we find two more con ports; there are four in total. Two more fan headers round out the connections for the board.

Looking at the back I/O ports, we can see the first stack consists of two USB 2.0 ports, followed by a serial port, and a VGA port just below. Next, we have a UID switch, and two LAN ports. The last stack has an IPMI LAN port at the top, and two USB 3.0 ports below.

Here we get an edge on view of the four con ports at the front of the motherboard.

Here we get a look at the new socket 2011-3; this one is the narrow version. The mounting holes will fit all your standard heat sinks.

BIOS, Software & Remote Management

BIOS

The BIOS for this motherboard is standard for server motherboards, so we will only show a few BIOS screens.

This is the main BIOS screen, which shows basic system information.

The Advanced tab brings you to the main advanced screen. Features we find here include ACPI Configuration, Configure Super IO Settings, Serial Port Console Redirection, CSM Configuration, USB Configuration, System Configuration, 3rd Storage Configuration, and H/W Monitor and Instant Flash.

Here are the Advanced 3rd Storage Configuration settings:

Marvell 9172 Controller:

Enable or disable Marvell 9172 Controller.

Marvell 9172 Operation Mode.

This item is for eSATA ports. Use this to select Marvell SATA operation mode.

Configuration options: [Disabled], [IDE Mode], [AHCI Mode], [RAID Mode] and

[Disabled]. The default value is [AHCI Mode].

Bootable Marvell 9172 SATA3

We recommend the use of Intel SATA ports (Port 0~5) for your bootable devices. This will minimize your boot time, and get the best performance. If you still want to boot from Marvell SATA3 controller, please set this item to Yes.

Here we are looking at the Intel RC Setup menu. This area includes configurations for Processor Configuration, CPU Power Management Configuration, QPI Configuration, Memory Configuration, IIO Configuration, PCH Configuration, and Server ME Configuration.

The first option is the Processor Configuration menu. Here you can adjust active cores, Hyper-Threading, no-execute memory protection, Intel TXT support, Intel Virtualization Technology, SMX, and more.

CPU Power Management menu brings us to this screen. Here you can turn off/on SpeedStep, and adjust other power states.

This is the QPI Configuration screen. Link Speed Mode can be changed to select the QPI link speed as the POR speed (Fast) or default speed (Slow).

The Memory Controller menu allows you to enforce POR restrictions for DDR4 and other settings.

Now we are looking at the PCH Configuration menu, and what settings that can be changed here.

The Boot menu will display the available devices on your system for you to configure the boot settings and the boot priority that you want to use.

Software

At the time of this review, we had a disk with only drivers on it; the disk was not complete at this time. When this board is released in retail, we are sure it will have a driver disk much like other ASRock Rack motherboards do.

Remote Management

Remote Access is supplied by the IPMI 2.0 with virtual media over LAN, and KVM over LAN support. Be sure to have a LAN cable plugged into the Dedicated IPMI LAN port.

We find our remote access IP address located in the BIOS under the IPMI Tab. In our case, this was 192.182.1.36. Enter the remote access IP into your browser and you will see the login screen.

To login use:

Username: admin

Password: admin

As a best practice, administrative users should change factory default Username/Password logins before connecting any new server to their network.

After logging in, we come to the home screen, and system information is displayed.

There is also a remote control option for iKVM.

The next tab is the FRU (Field Replacement Unit) menu.

Next up is the Server Health menu. This shows temperatures and voltages for the server.

The Configuration menu allows you to change many features on the server, including Active Directory settings, DNS, LDAP, and many more.

The Remote Control menu allows you to power on, shut down, restart, and cycle the server.

This page allows you to set triggers for events.

This page allows you to select a specific configuration to be preserved in case restore configuration is needed.

The Firmware Update menu allows you to update the firmware, and restore factory defaults.

Test System Setup

We would like to thank ASRock Rack,SPEC,Yokogawa,NZXT,SanDisk,Noctua, and Crucial for their support in providing parts for our test system.

The Haswell-EP and the Wellsburg Platform (Intel C612), with the new Intel Xeon E5-2698 v3 features 16 cores with Hyper-Threading used on these tests will supply the processing power.

The Wellsburg Platform (Intel C612) will have support provided for four to 18 cores with dual-socket capability. TDP ranges from 55W up to 160W for workstations. Memory is now DDR4, and can gave a frequency of up to 2133 MHz. The E5-2600 v3 processors use 2x QPI 1.1 channels with up to 9.6 GT/s. These processors support PCIe 3.0 with up to 8 GT/s and 40 lanes. The chipset will be Wellsburg PCH. This gives support for a huge number of SATA ports at ten. A large number of USB devices can be used with 6x USB 3.0, and 8x USB2 ports. Wellsburg (C612) also supports DMI2 with 4x lanes.

In our tests, we will be using the new Crucial DDR4 memory, which has a speed of 2133 MHz, and is rated at CL15. We have already taken a look at these memory kits, which you can find here: Crucial DDR4 Memory Performance Overview Early Look vs. DDR2 & DDR3.

Let's get busy with this new ASRock EP2C612D16FM motherboard, and see how the new platform performs. On all our tests, we used the default BIOS settings, and no other tweaking is used to run any tests shown.

System and CPU Benchmarks

CINEBENCH 11.5

CINEBENCH is a real-world, cross platform test suite that evaluates your computer's performance capabilities. The test scenario uses all of your system's processing power to render a photorealistic 3D scene. This scene makes use of various different algorithms to stress all available processor cores. You can also run this test with a single-core mode to give a single-core rating.

Using the new E5-2698 v3 16-core processor, we have a very nice score in CINEBENCH 11.5. As of this writing, the 43.02 score is the highest we have seen in the lab. The new Haswell-EP processor features, and the number of cores used, provide the number crunching power for this test. It's amazing just how much CPU power these new Haswell-EP processors brings to the table.

This is an impressive score coming out of the gate on our bench runs. We think this kind of performance may possibly carry on to the rest of the benchmarks.

CINEBENCH R15

Just like in the CINEBENCH 11.5 results, the E5-2698 v3's 16-core processors used on the ASRock EP2C612D16FM motherboard have also reached a new high score for us in CINEBENCH R15.

wPrime

wPrime is a leading multi-threaded benchmark for x86 processors that tests your processor performance. This is a great test to use to rate the system speed; it also works as a stress test to see how well the system's cooling is performing.

wPrime is one of those benches that can really stress the system, and is difficult to get low scores with. In this case, the ASRock EP2C612D16FM and E5-2698 v3's processors just nailed it, and not by a little. We see a clear lead of above ten seconds compared to last gen processors and setups. It also stands up very strong to the single-threaded runs. We see very impressive scores here.

Memory & UnixBench Benchmarks

AIDA64

AIDA64 memory bandwidth benchmarks (Memory Read, Memory Write, and Memory Copy) measure the maximum achievable memory data transfer bandwidth.

Now we get a look at how the Crucial DDR4 memory performs on the ASRock EP2C612D16FM motherboard. We see good improvements on bandwidth using DDR4, and it offers improvements in latency over DDR3, which we saw around 90-103ns. With DDR4, we see a big improvement, with latency dropping to around 77ns. Faster memory and lower latency with higher capacities offers higher performance for memory driven applications.

Linpack

Intel Optimized LINPACK Benchmark is a generalization of the LINPACK 1000 benchmark. It solves a dense (real*8) system of linear equations (Ax=b), measures the amount of time it takes to factor and solve the system, converts that time into a performance rate, and tests the results for accuracy.

LINPACK is a measure of a computer's floating-point rate of execution ability, measured in GFlops (Floating-point Operations per Second); 10 billion FLOPS = 10 GFLOPS. LINPACK is a very heavy computing application that can take advantage of the new AVX2 instruction. As it puts a very high load on the system, it is also a good stress test program.

The E5-2698 v3 processors and AVX2 really take off in this benchmark, with almost double the performance of E5-2697 v2 processors. We have also seen results as high as 800 GFlops using E5-2699 v3 processors.

We think LINPACK really shows the potential of the raw computational power these new platforms offer to the enterprise markets.

UnixBench 5.1.3

UnixBench has been around for a long time now, and is a good general-purpose benchmark to test on Linux-based systems. This is a system benchmark, and shows the performance of single-threaded and multi-threaded tasks.

UnixBench is also showing strong numbers in these tests. These numbers are not as high as we would like to see though. It is very possible that we will need an update to UnixBench to get this benchmark running in top shape.

SPEC CPU2006v1.2 Benchmark

SPEC CPU2006 v1.2

SPEC CPU2006v1.2 measures compute intensive performance across the system using realistic benchmarks to rate real performance.

In our testing with SPEC CPU2006, we use the basic commands to run these tests:

Runspec --tune=base --config=tweaktown.cfg, then int, or fp.

To do multi-threaded we add in --rate=64.

When SPEC CPU first came out, these tests could take up to a week to run, but as computers become faster, our tests now take up to four days for a full run, and even less on some systems. The user can do many thing to effect the results of CPU2006 runs, such as compiler optimizations, add-ons like Smartheap, and different commands used to start the tests.

This benchmark has many different commands to use depending on what the user is looking for. For our tests, we used basic commands that run a full test with a base tune. As you can see here, the SPEC scores after full runs for Integer (int) and Floating Point (fp) tests. Single-core runs show how fast (speed) a CPU can perform a given task. In the multi-core runs, we set SPEC CPU2006v1.2 to use all threads to measure the throughput of the system.

The additional cores/threads of this system have a huge impact on performance in these tests, and really shows the amount of horsepower that a dual-socket system has over a single-socket board. Single-threaded results are still very important, but when you need many single-threaded apps to run, moving to a CPU with more cores is the way to go.

In CPU2006, we start to crunch the new E5-2698 v3 processor hard. Heat was not much of an issue during these tests, and that shows this platform can handle stress very well. We see very promising scores on the ASRock EP2C612D16FM platform, which shows what the new E5-2600 v3 processors and DDR4 can do in crunching power.

Looking at the results of single-threaded integer runs, we can get an idea of speed at which the E5-2698 v3's can crunch through the different integer tests. Not all CPUs are equal here, and ones that have a higher speed will perform these tests faster. Naturally, using an overclocked system or CPUs with a higher stock speed will generate higher results.

Now we run the test using all 32 cores/64 thread cores on the E5-2698 v3 processors to measure the throughput of the system. In this test, more cores/threads will have a greater effect on the outcome.

Just like the integer tests, we now run the floating-point tests in single-threaded (speed) mode.

Here we see the results of the multi-core, floating-point run that uses all 32 cores/64 thread cores on the E5-2698 v3 processors. Like the multi-threaded integer test, more cores/threads will have a greater impact on the test.

Power Consumption & Final Thoughts

Power Consumption

We have upgraded our power testing equipment, and now use a Yokogawa WT310 power meter for testing. The Yokogawa WT310 feeds its data through a USB cable to another machine where we can capture the test results.

To test total system power use, we used AIDA64 Stability test to load the CPU, and then recorded the results. We also now add in the power use for a server from off state to hitting the power button to turn it on, and taking it all the way to the desktop. This gives us data on power consumption during the boot up process.

Power use on the new platform is also very good. Idle power use is below 100watts, which is outstanding for a dual-socket system. Peak power load reaches ~425watts, which is also very good.

Factors that affect this test include the amount of RAM installed, and the number of onboard options such as storage controllers, or network adapters. Per Core P-States (PCPS) allows cores to run at individual frequency/voltages. Energy efficient turbo mode (EET) monitors stall behavior and increases throughput. With Haswell-EP, each core and uncore, are now treated independently. Core bound applications can drive frequency higher without needing to increase uncore. LLC/Memory bound applications can drive frequency higher without burning core power. We can see the effects of PCP with the lower power draw of the system.

Booting a system up can cause a power surge if many servers start up at the same time. With the system off, we measured only ~4.5watts to keep IPMI active. When the system entered the boot process, power spikes to ~200watts, with a bump to ~225watts. After boot up has completed, the system settles down to sub 100watt levels.

Final Thoughts

This our first Intel Haswell-EP motherboard review, and we can say without a doubt that that this new platform has performance written all over it. The ASRock EP2C612D16FM nails this with a top performing motherboard, which loads it up with a large amount of storage features.

We are going to start seeing plenty of new Haswell-EP platforms in the test lab; this one was our first, and it knocked our socks off. There is so much computational power with these new processors and DDR4, that it makes upgrading a no brainer.

There are large amounts of storage and fast network options right on board now, which will handle basic system needs, and then some. We expect these systems with E5-2699 v3 18 core processors to be even more powerful.

In the data centers, systems like these can replace a number of pre-existing systems, which lowers costs even further from power use, and makes administration that much easier. Processors with more cores and new DDR4 with larger capacity memory sticks will work hand in hand with highly utilized VM systems. The new Haswell-EP architecture will take VM performance to the next level, and will be the next wave of data center systems.

Systems based on the new ASRock EP2C612D16FM are smaller, use less power, are higher performing than previous gen systems, and are set to take on the expanding needs for VMs and storage issues in the modern data center. Per Core P-States (PCPS) have also helped to reduce power usage on these new systems, which helps reduce running costs.

The EP2C612D16FM has easy to use BIOS, and offers many new features, but what is missing on the board is Cluster on Die (COD), which is located in the QPI settings. Not all motherboards need this feature, but this would help to increase performance, reduce latency, and improve performance by better application placement in a VM environment. COD is best used for highly NUMA (Non-uniform memory access) optimized workloads.