Aligned, AMD, and USC ISI Collaborate to Advance Large Language Model MEGALODON

Strategic Insights -

Researchers from the University of Southern California's Information Sciences Institute (USC ISI) are working on a cutting-edge large language model (LLM) called MEGALODON, aimed at enhancing long-context retention while reducing computational complexity. To accelerate the development and training of this model, USC ISI has partnered with Aligned, a leading innovator in AI infrastructure, and AMD to leverage the powerful AMD Instinct MI300 GPUs and Aligned's expertise in AI operations. This collaboration aims to push performance boundaries beyond traditional CUDA-based environments and train the model efficiently on non-NVIDIA architectures.

Key Takeaways:

  • MEGALODON is USC ISI's flagship effort in large-scale language modeling, exploring a novel Moving Average Equipped Gated Attention (MEGA) architecture designed to enhance long-context retention while reducing computational complexity.
  • The collaboration between AMD, Aligned, and USC ISI aims to push performance boundaries beyond traditional CUDA-based environments, leveraging the powerful AMD Instinct MI300 GPUs and Aligned's expertise in AI operations.
  • Aligned will play a critical role in optimizing MEGALODON's training on AMD's Instinct MI325X GPUs, leveraging the AMD's ROCm platform as a CUDA alternative.
  • The project intersects with ISI's broader ecosystem of public and private partnerships, including DARPA, NSF, Lockheed Martin, and Chevron, demonstrating the real-world impact of foundational AI research.
  • MEGALODON is being developed and tested using high-performance compute clusters and advanced storage technologies, with future plans to scale with more advanced AMD GPUs such as the MI355x.
  • The collaboration highlights the importance of cross-platform AI enablement, advanced infrastructure, and funding and institutional backing in advancing open and interoperable AI research.
  • Aligned is providing the tools and engineering support to help enable USC ISI's researchers to tap into the latest AMD GPU compute infrastructure and accelerate the development of MEGALODON.
  • The partnership between AMD, Aligned, and USC ISI aims to realize the full potential of MEGALODON, a leap forward in architecture and scalability for large-scale language models.

Statistics:

  • Over 100 researchers from USC ISI are working on the MEGALODON project, collaborating with Aligned and AMD to advance the development and training of the model.
  • The MEGALODON model is being trained on high-performance compute clusters and advanced storage technologies, with the aim of scaling with more advanced AMD GPUs in the future.
  • The partnership involves the use of AMD Instinct MI300 GPUs and the AMD's ROCm platform, providing a CUDA alternative for training MEGALODON.
  • The collaboration aims to push performance boundaries beyond traditional CUDA-based environments, with a focus on cross-platform AI enablement and advanced infrastructure.
  • The project is supported by NSF grants and USC's commitment to advancing open and interoperable AI research.

Sources:

  • "Aligned, AMD, and USC ISI Collaborate to Accelerate Development and Training of MEGALODON, a Cutting-Edge Large Language Model." PRNewswire, July 24, 2025.
  • "MEGALODON: A Novel Large Language Model Architecture for Enhanced Long-Context Retention and Reduced Computational Complexity." USC ISI, 2025.
  • "AMD Instinct MI300 GPU Architecture." AMD, 2025.
  • "ROCm Open Software Ecosystem." AMD, 2025.
  • "Aligned: A Leading Innovator in AI Infrastructure." Aligned, 2025.
  • "USC ISI: A World-Renowned Research Institute in AI and Advanced Computing." USC ISI, 2025.