Breakthrough in Artificial Intelligence: Researchers Develop Automated Pruning Framework for Large Language Models

Strategic Insights -

Researchers from Kasetsart University have made a significant advancement in artificial intelligence by developing an automated pruning framework for large language models. The new framework uses combinatorial optimization to reduce the size of these models, making them more efficient and less resource-intensive. The study, published in the journal AI, demonstrates the effectiveness of the framework in reducing the size of the Llama-3.1-70B model by 13.44% while maintaining a high level of accuracy.

Key Takeaways:

  • The researchers developed an automated pruning framework for large language models using combinatorial optimization.
  • The framework systematically explores the search space to identify the most optimal pruning configurations, removing redundant or non-contributory parameters.
  • The study used two optimization techniques, particle swarm optimization (PSO) and whale optimization algorithm (WOA), to evaluate their ability to navigate the search space efficiently.
  • The framework was able to reduce the model size of Llama-3.1-70B by 13.44% and 12.07% using PSO and WOA, respectively, while maintaining high levels of accuracy.
  • The researchers also integrated a post-process to recover model accuracy after pruning, with a final pruned model loss of 12.72% and 14.83% using PSO and WOA, respectively.
  • The study's findings have significant implications for the development of more efficient and effective large language models.
  • The research was conducted by Patcharapol Ratsapa, Kundjanasith Thonglek, Chantana Chantrapornchai, and Kohei Ichikawa from the Department of Computer Engineering, Faculty of Engineering, Kasetsart University.

Statistics:

  • The Llama-3.1-70B model was reduced in size by 13.44% using particle swarm optimization (PSO)
  • The model was reduced in size by 12.07% using whale optimization algorithm (WOA)
  • The final pruned model loss using PSO was 12.72%
  • The final pruned model loss using WOA was 14.83%

Sources:

  • Patcharapol Ratsapa, Kundjanasith Thonglek, Chantana Chantrapornchai, Kohei Ichikawa. Automated Pruning Framework for Large Language Models Using Combinatorial Optimization. AI 2025, 6(5), 96. doi: 10.3390/ai6050096 (free version available at https://doi-org.sdpl.idm.oclc.org/10.3390/ai6050096)
  • NewsRx. Kasetsart University Researchers Advance Knowledge in Artificial Intelligence (Automated Pruning Framework for Large Language Models Using Combinatorial Optimization). Robotics & Machine Learning. June 9, 2025; p 299.