Unified Modeling Language Code Generation from Diagram Images Using Multimodal Large Language Models

Strategic Insights -

Research has revealed a groundbreaking approach to generating executable Unified Modeling Language (UML) code from image-based UML diagrams. This innovative method uses a large multimodal language model to automate the code generation process, proposing a new approach to tackle the challenge of generating executable UML code from UML diagrams. The study demonstrated that domain-adapted multimodal large language models perform exceptionally well for UML code generation automation, achieving high accuracy and enabling the modernization of legacy systems.

Key Takeaways:

  • Researchers at the University of Oklahoma proposed a new approach to generate UML code using a large multimodal language model automatically.
  • Synthetic UML activity and sequence diagram datasets were created to train and test the model.
  • The experiments compared standard fine-tuning with LoRA techniques to optimize base models, measuring code generation accuracy across different model sizes and training strategies.
  • The best-performing model achieved a BLEU score of 0.779 and a SSIM of 0.942 on sequence diagrams.
  • The approach demonstrated the potential to modernize legacy systems and decrease the manual effort put into software development workflows.
  • The study utilized a domain-adapted multimodal large language model, which outperformed traditional models in UML code generation.
  • The research also highlighted the importance of using large-scale multimodal datasets for training and testing the model.

Statistics:

  • 0.779 BLEU score achieved by the best-performing model on sequence diagrams.
  • 0.942 SSIM achieved by the best-performing model on sequence diagrams.
  • The study utilized a domain-adapted multimodal large language model.
  • The approach demonstrated a 0% manual effort reduction in software development workflows.

Sources:

  • Unified modeling language code generation from diagram images using multimodal large language models. Machine Learning with Applications, 2025, p 100660.