Tech content trusted by users in North America and around the world
7,219 Reviews & Articles | 52,140 News Posts

Supermicro 7048GR-TR (Intel C612) Workstation Tower System Review

By: William Harmon | Servers in IT/Datacenter | Posted: Jan 23, 2015 3:10 pm
TweakTown Rating: 97%Manufacturer: Supermicro

Setting Up

 

Before we begin using our Xeon Phi 3120A coprocessors, or even installing them on the X10DRG-Q motherboard, we need to enable "Above 4G Decoding" in the BIOS to be able to use these cards. If you did not do this beforehand, and turned on the system to just let it run, at some point you will get an BIOS error screen. From here, you can head over to PCIe/PCI/PnP Configuration menu, and change the setting.

 

As we said before, we had a great deal of interest in these cards, so we wanted to walk through our experience setting and using the 3120As that we had. For a first time user, it can be a little intimidating and frustrating going through this, but once you wrap your head about around what is going on, it's pretty simple to do. Let's get started.

 

Intel does provide a Xeon Phi Coprocessor Quick Start Developers Guide for Windows that we will follow along with. You should download this guide, and become familiar with what we will be going over here.

 

At this point, we have our system installed and running Windows 8.1 Enterprise. 4x Xeon Phi 3120As are installed in the PCIe slots, and the system is up and running. We will need to install drivers for the Xeon Phi, so head over to the Intel Developer Zone Page, Tools and Downloads for mic developers, and download the Manycore Platform Software Stack (MPSS).

 

You are looking for the latest MPSS release for Microsoft Windows. Inside this download, you will find two install packages that you will need to install, the Intel(R) Xeon Phi(TM) coprocessor, and the Intel(R) Xeon Phi(TM) coprocessor essentials.

 

You are also going to need a few other programs, so let's get those while we are at it. These include

Putty, Puttygen, and

WinSCP. You will also need Intel Composer XE 2015, so download and install this.

 

Let's also download Linpack_11.2.1 for our testing. Create a folder on your desktop called "winshare," and put the unzipped linpack_11.2.1 folder in that.

 

We also need to find another file called "libiomp5.so." This is located in the Intel Composer XE 2015 directory, C:ProgramFiles (x86)IntelComposer XE 2015compilerlibmic. Just copy "libiomp5.so" (without the quotations), and paste it into the winshare folder.

 

Right now, we should have all of the software we need downloaded and installed on our test system, and the two files placed inside the winshare folder. Make sure you have completed all these steps before you continue.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_59

 

After you have installed both Intel(R) Xeon Phi(TM) coprocessor and Intel(R) Xeon Phi(TM) coprocessor essentials packages, bring up your control panel, and head to Programs and Features to confirm these two packages are installed. It should look similar to the picture above.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_60

 

Now, bring up the Device Manager to check if the Xeon Phi cards show up. You can see we show 4x Xeon Phi cards here. So far, so good. It's fairly simple.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_61

 

Now what we want to do is open your C: drive, drill down into C:Program FilesIntelMPSSbin, and find "micsmc-gui." You can create a shortcut for this, and place it on your desktop. Go ahead and double click it. What you will see is the top window, which is the MPSS control panel. It will display average temps, memory usage, how many watts are being used, and average core utilization.

 

The "Cards" button will drop down a list of all the Xeon Phi cards installed. They will be listed as mic0, mic1, mic2, mic3 for our system. This can show all of the cards, or just the main display. The next button is "Advanced," which will show error logs, card info, and card settings. Here you can reconnect or restart your cards.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_62

 

Under the "Cards" button, if we select "Show All," we will see this screen. The top window shows total system use, followed by each card below. Great. So far, we have our cards up and running, and we can see what each one is doing.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_63

 

Now let's bring up a command line window, and run it as Administrator.

 

At the prompt, enter ">micinfo".

 

Now we will see complete information on all four cards. In this command window, we can issue many different commands to manage our cards.

 

Go ahead and enter ">micctrl -stop".

 

This will stop the cards.

 

Issue ">micctrl -start".

 

This will start the cards up.

 

At this point, find the Putty and Puttygen that you downloaded before, and place them into C:Program FilesIntelMPSSbin. We will use Putty to issue commands to the Phi cards via the SSH command window. Puttygen will create public and private SSH keys for us, which we will need to use for a secure connection.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_65

 

Go ahead and run Puttygen. We used 1024 for number of bits in our key to make a shorter key. When you select "Generate" move your mouse around in the window above to create randomness to generate the key.

 

Your screen should look something like this when done.

 

Open notepad and create a file called "authorized_keys" (no .txt file extension). Copy the Public Key string, and paste it into this file. When finished, place that file into C:Program FilesIntelMPSSbin

 

Now hit "Save Private Key," give it a name "id_rsa.ppk," and save it into C:Program FilesIntelMPSSbin.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_64

 

The C:Program FilesIntelMPSSbin folder should look just like this now.

 

Next, we are going to run Putty to open a SSH connection to the Xeon Phi cards. We did have some issues with this; for some reason it did not like our "authorized_keys" file.

 

To get around this issue, we used this command in our Windows' command line window.

 

> micctrl --addssh root "your key that you generated in Puttygen"

 

You can open the "authorized_keys" file, and copy the key there to use in this command line.

 

Issuing this command will upload your key to each of your cards.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_66

 

Launch Putty, and you will see this screen.

 

For Host Name use:

root@192.168.1.100

 

This will be the first card, or mic0. For mic1, mic2, mic3 use:

 

mic1 - root@192.168.2.100

mic2 - root@192.168.3.100

mic3 - root@192.168.4.100

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_67

 

Move down to SSH, and open that selection. Then select "Auth" to see the screen above.

Here, hit the "Browse" button, and locate the "id_rsa.ppk" file. It will be located in the directory shown in the screen shot.

 

Then hit "Open."

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_68

 

Now we have an open SSH window into the root level of mic0. Here is where we had problems, it would ask for a password and would not open for us. This is why we used the "> micctrl --addssh root 'your key that you generated in Puttygen'" command before.

 

Once we issued that command, we had no problems, and the Putty SSH command window started just as we see here. So far, so good; we now have an open SSH connection where we can issue SSH commands to the Xeon Phi card. Next, you are going to want to download and install WinSCP, if you have not done so already.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_69

 

Before we can use WinSCP, we will need to generate a password. In the Putty SSH command window, enter:

 

>passwd

 

Enter a password that you will use for WinSCP.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_71

 

Are you with us so far? Good. Now we are going to use WinSCP to upload LINPACK and the libiomp5.so file to the Xeon Phi cards. Launch WinSCP and change the file protocol to SCP. Enter the IP address for your Xeon Phi card, which is 192.168.1.100.

 

In the password field, enter the password you created in the Putty SSH command window with >passwd.

 

You can now save this, so you do not have to do it again. Notice we have done this for all four Xeon Phi cards in our system. Log in when completed.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_72

 

WinSCP will start and open a window, just as we show here. In the left window, navigate to the "winshare" folder you created on your desktop. The right window will be looking at the root directory of the Xeon Phi card. Hit the top icon, or back one directory.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_73

 

Now we are looking at all the system folders on the Xeon Phi in the right window. Looks just like a Linux machine, right?

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_74

 

Open the "lib64" folder, and upload the "libiomp5.so" file into it.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_75

 

Let's go back one directory in the right window, and open the "Home" folder. Here you can see the list of users on the Xeon Phi. The "micuser" folder was created by us before. Upload the "linpack_11.2.1" folder now.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_76

 

Now, let's open the "linpack_11.2.1" folder, and move down the directory to the "linpack" folder; you can see the listing of files in it here. If you try to run LINPACK now, you will receive permission errors, so we need to fix that.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_70

 

Go back to the Putty SSH command window, and change the directories to the LINPACK folder.

 

Enter: # cd /home/linpack_11.2.1/benchmarks/linpack

Then: # ls

 

We now see a listing of files in the "LINPACK" folder. We need to change permissions on two files. Enter the following commands:

 

# chmod 777 xlinpack_mic

# chmod 777 runme_mic

 

Great, we are almost ready to run our first test.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_77

 

The nest problem we ran into was with running LINPACK.

The downloaded version of LINPACK is created for a different setup with 16GB of RAM on the Xeon Phi cards, and will not work on the 3120A, which only has 6GB of RAM. Let's fix that. Select the "lininput_mic" file, and right click to bring the menu up, then select "Edit."

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_78

 

This is what you will see after you hit "Edit." The two lines we need to fix are the long numbered ones labeled "# problem sizes" and "# leading dimensions."

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_79

 

Remove the last four group of numbers in each of those. Replace them with:

 

23000

23024

 

Those two edited lines should now look just like what is shown above. Hit the "Save" button.

 

Wow, that was a lot of work to get these cards setup; I hope you were able to follow along, and get to this point without too many issues. We are now ready to run our first test.

 

 

Testing

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_80

 

Let's go back to our desktop and look at the MPSS control panel, and have it show the first Xeon Phi card, or mic0. We will now see two windows, one for total system use, and mic0 just below that.

 

Back in our Putty SSH command window, we only have to enter one command to start LINPACK:

# ./runme_mic

 

The window now shows LINPACK running on one Xeon Phi card. You can see Core Utilization graph showing different CPU loads, temperatures should increase a bit, and watt usage should also increase.

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_81

 

After about 15 to 20 minutes, LINPACK will finish up ,and we can see our test results. Wow - look at that! 713 GFLOPS on one card. That is impressive!

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_82

 

Because we wanted to fully load our test system to measure power use, we ran LINPACK on all 4x Xeon Phi cards. We are looking at ~2,852 GFLOPS right now, which is very impressive!

 

supermicro-7048gr-tr-intel-c612-workstation-tower-system-review_83

 

Even though we only have one test, we are going to chart this result, so we can use it later when we test other Xeon Phi cards.

    PRICING: You can find products similar to this one for sale below.


    United States: Find other tech and computer products like this over at Amazon's website.

    United Kingdom: Find other tech and computer products like this over at Amazon UK's website.

    Canada: Find other tech and computer products like this over at Amazon Canada's website.

    We at TweakTown openly invite the companies who provide us with review samples / who are mentioned or discussed to express their opinion of our content. If any company representative wishes to respond, we will publish the response here.

Related Tags

Got an opinion on this content? Post a comment below!
loading