GSI Technology's Associative Processing Unit (APU) Achieves Breakthrough in AI Inference Processing

Strategic Insights -

Research led by Cornell University has validated the performance of GSI Technology's Associative Processing Unit (APU) in delivering GPU-class performance for large-scale AI applications with significant energy reduction. The study, titled "Characterizing and Optimizing Realistic Workloads on a Commercial Compute-in-SRAM Device," confirmed that the Gemini-I APU can match NVIDIA's A6000 GPU performance on RAG workloads, while consuming over 98% less energy.

Key Takeaways:

  • The Gemini-I APU delivered comparable throughput to NVIDIA's A6000 GPU on RAG workloads, demonstrating its potential to replace traditional GPUs in AI inference applications.
  • The APU achieved a 98% reduction in energy consumption compared to GPUs over various large corpora datasets, highlighting its efficiency and sustainability in power-constrained environments.
  • The APU's unique design enabled it to perform retrieval tasks several times faster than standard CPUs, shortening total processing time by up to 80%.
  • The research points to significant opportunities for GSI Technology as customers increasingly require performance-per-watt gains across various industries, including Edge AI, defense, and aerospace applications.
  • The Cornell study introduced a new analytical framework for general-purpose compute-in-memory devices, providing optimization principles that strengthen the APU's position as a scalable platform for developers and system integrators.

Statistics:

  • 98%: the energy consumption reduction achieved by the Gemini-I APU compared to GPUs over various large corpora datasets.
  • 80%: the reduction in total processing time achieved by the APU's unique design compared to standard CPUs.
  • 10x: the increase in throughput and lower latency achieved by the Gemini-II APU silicon compared to the Gemini-I APU.

Sources:

  • "Characterizing and Optimizing Realistic Workloads on a Commercial Compute-in-SRAM Device" (ACM paper)
  • GSI Technology's press release announcing the publication of the paper
  • GSI Technology's website
  • GSI Technology's investor relations contact information.