SanDisk ULLtraDIMM Internals and Specifications
The ULLtraDIMM connects into an industry-standard 240-pin DIMM slot, and features a LP 30mm x 133.35 RDIMM form factor. The DIMM is compatible with the JEDEC DDR3 protocol, and operates at the full DDR bus rate, from DDR3-800 through DDR3-1600. The device follows the same motherboard population rules that apply to standard DRAM sticks, and all communication with the host occurs on the memory bus. The device is interoperable with standard RDIMMs in the same channel, and appears as a single rank to the system. The ULLtraDIMM is compatible with Linux (RHEL, SLES, CentOS), Windows, and VMWare ESXI.
Many servers already have spare DRAM slots, especially in comparison to the number of available PCIe slots. There are up to 96 slots available on the memory bus. SanDisk and Diablo are also developing new DDR4 ULLtraDIMMs. With new high-density DDR4 128GB DRAM sticks, there is flexibility to deploy plenty of DRAM and ULLtraDIMMs to satiate even the most demanding workload requirements.
The close orientation of the devices illustrates the limited real estate on a typical motherboard. The memory slots have such limited space due to the high-speed parallel bus, which must be in close proximity to the CPU because it cannot transmit over long distances. PCIe is a serial bus, allowing it to be further from the CPU, but has an overall bandwidth limitation of 16 GB/s. PCIe bandwidth is carved up into an allocation for each lane, meaning a 16-lane device effectively has a top speed of 8 GB/s. A single DRAM slot, in contrast, can transfer 12.8 GB/s at nanosecond latencies.
There are no physical modifications of existing hardware for ULLtraDIMM compatibility, but UEFI BIOS changes are required. The BIOS modifications disable memtest, reserve memory space for the device, and create an entry for the ULLtraDIMM in the ACPI table. ULLtraDIMMs are designed to fit within the power and thermal envelopes of the existing memory subsystem. The 12.5W power draw and heat dissipation are well within the tolerance of a standard server. The ULLtraDIMMs' vertically aligned heat sinks take advantage of the server's native linear airflow pattern.
We tested the ULLtraDIMMs on a Supermicro X9DR3-LN4F+ development system. The X9DR3-LN4F+ features the C606 chipset, supports the E5-2600 v2 family, and up to 1.5TB of ECC DDR3 DRAM. The system also features six PCIe slots and 4x Gigabit Ethernet ports via an Intel i350 controller. 2x SATA 6Gb/s ports, 4x SATA 3Gb/s ports, and 6x SAS ports round out an impressive stable of connectivity accentuated with ULLtraDIMM compatibility.
Many of the surface-mounted components of the actual device are covered by heat sinks. The large SanDisk blocks in the illustration are dual Marvell 88SS9187 controllers that control the two banks of NAND on the other side of the device. There is a 1GB Micron DRAM package for each respective SSD ASIC next to the NAND. Each configuration is built from CloudSpeed 1000E components (covered here).
Each module features 16 independent channels into the DIMM. The nine red boxes along the bottom of the ULLtraDIMM, and the large chip in the center, are Diablo components. The DDR3 interface layer provides the electrical and timing interface. The Diablo chipset is responsible for capturing data from the host management engine by decoding host commands, and deciphering packets. Utilizing on-board ASICs keeps host overhead low by removing the lion's share of processing burden from the host.
The six PolyTantalum capacitors, which are an integral part of the third-generation EverGuard design, provide power-loss protection for the entire device, including the MCS chipset. This guarantees all data will be persisted to the underlying media in the event of a power-loss event. A debug port (not used in deployment) is located on the top of the ULLtraDIMM.
The ULLtraDIMM is JEDEC-compliant with JC-45-2065.45 and JESD218 standards.
PRICING: You can find products similar to this one for sale below.
United States: Find other tech and computer products like this over at Amazon's website.
United Kingdom: Find other tech and computer products like this over at Amazon UK's website.
Canada: Find other tech and computer products like this over at Amazon Canada's website.
- Page 1 [Introduction]
- Page 2 [Internals and Specifications]
- Page 3 [MCS Management Console]
- Page 4 [MCS Architecture and VSAN]
- Page 5 [Guardian Technology Platform]
- Page 6 [Test System and Methodology]
- Page 7 [Benchmarks - 4k Random Read/Write]
- Page 8 [Benchmarks - 8k Random Read/Write]
- Page 9 [Benchmarks - 128k Sequential Read/Write]
- Page 10 [Database/OLTP and Webserver]
- Page 11 [Email Server]
- Page 12 [Final Thoughts]