Microsoft Research roughly triples amount of data sorted in 60 seconds while using less hardware

Microsoft has set a new record in the MinuteSort test, nearly tripling the previous record from 2009.

@tracehagan
Published Mon, May 21 2012 3:31 PM CDT   |   Updated Tue, Nov 3 2020 12:28 PM CST

Microsoft has announced a victory in the MinuteSort test. They claim to have tripled the amount of data sorted by the previous record holder, a Yahoo team. MinuteSort is a test to see how much data can be sorted in just a mere 60 seconds. As more data moves into the cloud, this ability to sort data quickly becomes a bigger and bigger issue.

Microsoft Research roughly triples amount of data sorted in 60 seconds while using less hardware | TweakTown.com

According to Microsoft's post on TechNet, "In raw numbers, the team's system sorted 1401 gigabytes in just 60 seconds - using 1033 disks across 250 machines." This hardware compared to what Yahoo ran is roughly "one-sixth of the hardware resources" and managed to sort around 3 times as much data. You can see that the Microsoft solution is much more efficient.

Additionally, it's interesting to note that Microsoft Research didn't use Hadoop as one might expect. Instead, the researchers at Microsoft created a new system called "Flat Datacenter Storage." The "flat" portion is the important part of the system. Microsoft explains:

[Microsoft Research's Jeremy] Elson compares FDS to an organizational chart. In a hierarchical company, employees report to a superior, then to another superior, and so on. In a "flat" organization, they basically report to everyone, and vice versa.

Along with "full bisection bandwidth networks," this FDS has proved to be an effective way to sort data. Hadoop currently has the industry as a commercial standard, so FDS may take some of this thunder. Microsoft has hinted that it will likely be deployed on some of its projects. The numbers have not been confirmed by an external entity yet, so, for now, this is just Microsoft praising itself. I suggest taking it with a grain of salt until someone else confirms the numbers.

NEWS SOURCE:thenextweb.com

Trace is a starving college student studying Computer Science. He has a love of the English language and an addiction for new technology and speculation. When he's not writing, studying, or going to class, he can be found on the soccer pitch, both playing and coaching, or on the mountain snowboarding.

Newsletter Subscription

Related Tags

Newsletter Subscription
Latest News
View More News
Latest Reviews
View More Reviews
Latest Articles
View More Articles