Samsung has announced out of nowhere that it is manufacturing 24Gb DDR5 memory chips, which will allow for single-stick 768GB DDR5 DIMMs... yes 768GB of RAM on a single stick.
The company has already shown 512GB registered DIMM (RDIMM) that uses 32 x 16GB stacks, based on 8 x 16Gb DRAM chips. Low power and quality signaling is done by the 8-Hi stacks through silicon via interconnects, but with using 24Gb memory ICs in 8-Hi stacks, Samsung can boost one up to 24GB on a 32-chip module to an insane 768GB.
Inside of a server system with 2 modules per channel, we will see a world of 12TB of DDR5 memory. Right now, Intel's current Xeon Scalable "Ice Lake-SP" CPU is designed for these exact workloads: memory, memory, memory and they only support up to 6TB of RAM right now.
During their recent earnings call, a Samsung executive said: "In order to meet the demand and request by the cloud companies, we are also developing a maximum 24Gb DDR5 product".
Samsung won't just be making these huge insane 768GB DIMMs, but the company could also make 96GB, 192GB, and 384GB memory modules. On the client/consumer side of things, the 24Gb memory chips will usher in 24GB and 48GB DDR5 DIMMs, which would ensure HEDT systems could have 128GB+ of DDR5 very easily.
There's no ETA on when Samsung will start production, or when theses huge new DDR5 modules will be available.