Sign in Subscribe
Machine learning

Breakthrough Patent Application for Transfer Semantic Segmentation Via Learnable Image Prompting of Foundation Models

Strategic Insights

— 2 min read

Researchers have filed a patent application for a groundbreaking system and method that enables the transfer of semantic segmentation via learnable image prompting of foundation models. The innovative approach aims to overcome the challenges of conventional methods, which often rely on pseudo-labeling and suffer from confidence bias and high computational costs. The patented system utilizes a foundation model associated with a machine-learning network, receiving one or more fixed text prompts and one or more images, and outputting an intermediate representation from generating a series of objects and a task. The method then decodes the intermediate representation utilizing a decoder associated with the foundation model to generate a matrix associated with the task, and finally outputs a final label associated with a visual-based prediction task.

Key Takeaways:

  • The patented system and method enable the transfer of semantic segmentation via learnable image prompting of foundation models, overcoming the challenges of conventional methods.
  • The patented system utilizes a foundation model associated with a machine-learning network, receiving one or more fixed text prompts and one or more images.
  • The method decodes the intermediate representation utilizing a decoder associated with the foundation model to generate a matrix associated with the task, and finally outputs a final label associated with a visual-based prediction task.
  • The patented system includes a controller configured to receive one or more fixed text prompts and one or more images, and in response to utilizing the fixed text prompt and the one or more images at a foundation model associated with a machine-learning network, output an intermediate representation from generating a series of objects and a task.
  • The patented system further includes using a fusion model to combine representations from the foundation model with task-specific representations from an encoder.
  • The patented system has numerous applications, including semantic segmentation, visual-based prediction tasks, and multimodal models.
  • The patented system and method can be implemented in various domains, including computer vision, machine learning, and medical imaging.

Statistics:

  • The patented system and method aim to overcome the challenges of conventional methods, which often rely on pseudo-labeling and suffer from confidence bias and high computational costs.
  • The patented system utilizes a foundation model associated with a machine-learning network, which enables the transfer of semantic segmentation via learnable image prompting.
  • The patented system includes a controller configured to receive one or more fixed text prompts and one or more images, and in response to utilizing the fixed text prompt and the one or more images at a foundation model associated with a machine-learning network, output an intermediate representation from generating a series of objects and a task.
  • The patented system further includes using a fusion model to combine representations from the foundation model with task-specific representations from an encoder.

Sources:

  • DAS, Rajshekhar; FRANCIS, Jonathan; KULKARNI, Tanmay; MEHTA, Sanket Vaibhav. System And Method For Transfer Semantic Segmentation Via Learnable Image Prompting Of Foundation Models. U.S. Patent Application Number 20250200928, filed December 14, 2023 and posted June 19, 2025. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(20250200928)&db=US-PGPUB&type=ids
  • NewsRx LLC, "Breakthrough Patent Application for Transfer Semantic Segmentation Via Learnable Image Prompting of Foundation Models" (2025).

Read more