Micron Technology Announces Customer Sampling of 192GB SOCAMM2 for AI Data Centers

Strategic Insights -

As the technology industry continues to experience unprecedented growth in AI innovation, companies are shifting their focus towards sustainable and energy-efficient solutions. Micron Technology, Inc. has announced customer sampling of its SOCAMM2, a 192GB small outline compression attached memory module (SOCAMM2) designed to provide low-power memory solutions for AI data centers. This move aims to accelerate the industry's transition towards more energy-efficient infrastructure, reducing time to first token (TTFT) by over 80% in real-time inference workloads.

Key Takeaways:

  • Micron Technology, Inc. has announced customer sampling of its SOCAMM2, a 192GB small outline compression attached memory module, to enable broader adoption of low-power memory within AI data centers.
  • SOCAMM2 delivers 50% more capacity in the same compact footprint as its predecessor, using Micron's most advanced 1-gamma DRAM process technology to deliver a greater than 20% improvement in power efficiency.
  • The added capacity can significantly reduce TTFT by more than 80% in real-time inference workloads, with savings becoming quite significant in full-rack AI installations, which can include more than 40 terabytes of CPU-attached low-power DRAM main memory.
  • Building on a five-year collaboration with NVIDIA, Micron pioneered the use of low-power server memory in the data center.
  • SOCAMM2 provides the high data throughput required for AI workloads while delivering new levels of energy efficiency, setting a new standard for AI training and inference systems.
  • Through specialized design features and enhanced testing, Micron SOCAMM2 products transform low-power DRAM, initially designed for mobile phones, into data center-class solutions.

Statistics:

  • 192GB capacity per module
  • Speeds up to 9.6 Gbps
  • Power efficiency improvement of more than two-thirds compared to equivalent RDIMMs
  • Improved serviceability through modular design and innovative stacking technology
  • Time to first token (TTFT) reduction of more than 80% in real-time inference workloads

Sources:

  • GLOBE NEWSWIRE, "Micron Technology, Inc. Announces Customer Sampling of 192GB SOCAMM2 for AI Data Centers"
  • Micron Technology, Inc., "Micron Technology, Inc." About Us page
  • Micron Product and Technology Communications Contact: Mengxi Liu Evensen +1 (408) 444-2276
  • Micron Investor Relations Contact Satya Kumar +1 (408) 450-6199