NTT Research Presents 12 Papers at ICML 2025 Conference

NTT Research's Physics of Artificial Intelligence (PAI) Group presented 12 papers at the 42nd International Conference on Machine Learning (ICML) in Vancouver, July 13-19, 2025. The conference focused on the advancement of machine learning, a branch of artificial intelligence. The papers explored various aspects of machine learning, including large language model accuracy, machine learning interpretability, and the neural mechanisms of short-term memory in recurrent neural networks (RNNs).

Key Takeaways:

• Researchers from NTT Research's PAI Group explored the phenomenon of "representation shattering" in large language models, which occurs when algorithms alter the models' weights, inadvertently affecting representations of entities beyond the targeted one, distorting relevant structures that allow a model to infer unseen knowledge about an entity.
• A fundamental limitation in Sparse Autoencoders (SAEs), dictionary learning frameworks for improved machine learning interpretability, was revealed. The researchers presented a solution, Archetypal SAEs (and a variant, Relaxed Archetypal SAEs), that significantly enhances SAE stability.
• Researchers provided new insights into short-term memory mechanisms in RNNs, proposing experimentally testable predictions for further systems neuroscience study.
• NTT Research and NTT R&D presented eight additional papers at ICML 2025, exploring topics such as portable reward tuning, plausible token amplification, and kernel intensity estimators for inhomogeneous Poisson processes.
• Researchers demonstrated a major improvement in the computational efficiency of large-data-set Poisson processes, used to analyze and forecast event patterns in space and time, from posts on social media platforms SNS to disease outbreaks.
• NTT is committed to developing AI technologies that enable sustainable development, respect human autonomy, ensure fairness and openness, and protect security and privacy.

Statistics:

• 32 papers presented at ICML 2025 were co-authored by scientists from NTT Research and NTT R&D.
• 12 papers were accepted by the PAI Group, exploring various aspects of machine learning, including large language model accuracy, machine learning interpretability, and neural mechanisms of short-term memory in RNNs.
• 8 papers presented at ICML 2025 were co-authored by scientists from NTT R&D laboratories in Japan.
• 5 additional papers authored or co-authored by NTT R&D scientists included topics such as positive-unlabeled AUC maximization, natural perturbations for black-box training of neural networks, and learning to generate projections.

Sources:

• 1. "Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing,"1 ICML 2025
• 2. "Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models,"2 ICML 2025
• 3. "Dynamical Phases of Short-Term Memory Mechanisms in RNNs,"3 ICML 2025
• 4. "Portable Reward Tuning: Towards Reusable Fine-Tuning Across Different Pretrained Models,"4 ICML 2025
• 5. "Plausible Token Amplification for Improving Accuracy of Differentially Private In-Context Learning Based on Implicit Bayesian Inference,"5 ICML 2025
• 6. "K2IE: Kernel Method-based Kernel Intensity Estimators for Inhomogeneous Poisson Processes"6 ICML 2025