EvoContext: A Novel Approach to Hyperparameter Optimization in Machine Learning

Researchers at Guizhou Normal University have introduced a new method called EvoContext, which aims to address the challenges of hyperparameter optimization in machine learning by leveraging genetic algorithms and large language models. According to the study, existing methods rely heavily on repetitive configurations and few-shot learning, which can limit the optimization process to local regions. EvoContext proposes to generate configurations that differ significantly from examples via external interventions, thereby breaking the self-reinforcing effect and enabling a more efficient approximation of the global optimum. The researchers conducted experiments on several real-world datasets, showing that EvoContext outperforms traditional and other LLM-driven approaches on HPO.

Key Takeaways:

• EvoContext is a novel approach to hyperparameter optimization in machine learning, which uses genetic algorithms and large language models to generate configurations that differ significantly from examples.
• The method involves two phases: initial example generation through cold or warm starting, and iterative optimization that integrates genetic operations for updating examples to enhance global exploration capabilities.
• EvoContext employs LLMs in-context learning to generate configurations based on competitive examples for local refinement.
• The research concluded that experiments on several real-world datasets show that EvoContext outperforms traditional and other LLM-driven approaches on HPO.
• EvoContext was supported by the National Natural Science Foundation of China (NSFC), National Key Research & Development Program of China, Guizhou Provincial Major Scientific and Technological Program, Guizhou Provincial Program on Commercialization of Scientific and Technological Achievements, and Research Projects of the Science and Technology Plan of Guizhou Province.
• The research team included Hui Li, Yutian Xu, Guozhong Qin, Panfeng Chen, Mei Chen, Yanhao Wang, Xibin Wang, and Wei Zhou from Guizhou Normal University.

Statistics:

• The research found that EvoContext outperforms traditional and other LLM-driven approaches on HPO in experiments on several real-world datasets.
• EvoContext was supported by 5 financial institutions in China.
• The research team consisted of 8 authors from Guizhou Normal University.
• The study was published in the journal Electronics in 2025.

Sources:

• NewsRx. New Electronics Study Findings Have Been Reported by Investigators at Guizhou Normal University (Evocontext: Evolving Contextual Examples By Genetic Algorithm for Enhanced Hyperparameter Optimization Capability In Large Language Models). Mathematics Week. July 8, 2025; p 301.
• Evocontext: Evolving Contextual Examples By Genetic Algorithm for Enhanced Hyperparameter Optimization Capability In Large Language Models. Electronics, 2025;14(11).
• Guizhou Normal University, College of Computer Science and Technology, State Key Lab Publ Big Data, Guiyang 550025, People's Republic of China.