Sign in Subscribe
Machine learning

Patent for Parameter-Efficient Method for Training Neural Networks Unveiled

Strategic Insights

— 2 min read

Researchers at Universitat Zurich have developed a novel parameter-efficient method for training neural networks, which tackles the challenge of adapting neural networks to specific tasks with minimal memory requirements. The invention, patented as US Patent Number 12400117, proposes a method called kernel modulation, which significantly reduces the number of parameters required for task adaptation.

This innovative approach involves freezing the weight parameters of a base neural network and using a separate lightweight kernel modulator network to adapt the frozen parameters. The kernel modulator network is much smaller than the base network, requiring only 1% to 2% of the base network parameters, resulting in a 50x to 100x reduction in memory footprint. The accuracy of the new task adaptation is improved compared to previous methods.

The patent describes a method of adapting a given neural network for a specific data classification task, which involves freezing at least some of the weight parameters of the base network, duplicating the frozen parameters, and applying a second artificial neural network to the duplicated parameters to obtain modulated parameters. The modulated parameters are then used to adapt the base network for the specific task.

Key Takeaways:

  • The invention proposes a parameter-efficient method for training neural networks, utilizing kernel modulation to adapt frozen weight parameters.
  • The method involves freezing the weight parameters of a base network, duplicating the frozen parameters, and applying a second artificial neural network to the duplicated parameters to obtain modulated parameters.
  • The kernel modulator network is significantly smaller than the base network, requiring only 1% to 2% of the base network parameters, resulting in a 50x to 100x reduction in memory footprint.
  • The accuracy of the new task adaptation is improved compared to previous methods.
  • The method is particularly useful for adapting neural networks to specific tasks with minimal memory requirements.
  • The kernel modulation approach has the potential to revolutionize the field of neural networks and their applications in AI and machine learning.
  • The invention is patented as US Patent Number 12400117 and has been published online on August 26, 2025.

Statistics:

  • The kernel modulator network requires only 1% to 2% of the base network parameters, resulting in a 50x to 100x reduction in memory footprint.
  • The accuracy of the new task adaptation is improved compared to previous methods.
  • The method is particularly useful for adapting neural networks to specific tasks with minimal memory requirements.
  • The kernel modulation approach has the potential to revolutionize the field of neural networks and their applications in AI and machine learning.

Sources:

  • Hu, Yuhuang. Parameter-efficient method for training neural networks. U.S. Patent Number 12400117, filed October 5, 2022, and published online on August 26, 2025.
  • Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(12400117)&db=USPAT&type=ids

Read more