Contrastive Reinforcement Learning-Based Navigation in Medical Imaging

Strategic Insights -

Scientists have been working on developing a method to aid medical imaging navigation using artificial intelligence. This technology aims to alleviate the problem of skilled sonographers and technicians being in high demand. The current approach uses a simulation environment, where a synthetic image is generated from a CT scan. However, this method lacks generalizability and is less suitable for interventional cases. The new approach, patented by Amadou et al., utilizes a contrastive reinforcement learning framework to train the AI. The inventors provide a detailed description of their method, including its components and the process of training the AI.

Key Takeaways:

  • The patented method, titled "Contrastive Reinforcement Learning-Based Navigation In Medical Imaging," uses a contrastive reinforcement learning framework to train the AI.
  • The inventors propose a method for navigation of an ultrasound imaging transducer, receiving a vector representing a goal for a position of the transducer.
  • The method generates an action to reposition the transducer based on the vector, using a processor input of the vector to a contrastive reinforcement learned policy network.
  • The action is generated by a neural network trained with an actor loss based on a critic network outputting a reward.
  • The method is trained to maximize the similarity between the state-action pair and the goal and minimize the similarity when sampled from a different trajectory.
  • The system for medical sensor navigation includes a memory configured to store a policy network machine trained in a contrastive reinforcement learning framework.
  • The policy network was trained using trajectories with inputs sampled from different patients for a same iteration in optimization of a critic network of the contrastive reinforcement learning framework.

Statistics:

  • The patent application number is 20250248684, filed February 5, 2024, and posted August 7, 2025.
  • The method was trained using trajectories from simulation from computed tomography or magnetic resonance imaging.
  • The policy network was trained using trajectories with inputs sampled from 5 different patients for a same iteration in optimization of a critic network.
  • The method has a contrastive reinforcement learning framework with a critic network comprising first and second encoders.

Sources:

  • Amadou, Abdoul Aziz; Ghesu, Florin-Cristian; Kim, Young-Ho; Rhode, Kawal; Sharma, Puneet; Singh, Vivek; Young, Alistair. Contrastive Reinforcement Learning-Based Navigation In Medical Imaging. U.S. Patent Application Number 20250248684, filed February 5, 2024 and posted August 7, 2025.
  • https://ppubs.uspto.gov/pubwebapp/external.html?q=(20250248684)&db=US-PGPUB&type=ids