Sign in Subscribe
Data mining

Automated Generation of Research Workflows from Academic Papers: A Technical Framework

Strategic Insights

— 1 min read

Researchers at the Nanjing University of Science and Technology have proposed a novel end-to-end framework for automatically generating comprehensive research workflows from full-text academic papers. The framework, which utilizes Natural Language Processing (NLP) techniques and machine learning algorithms, has been tested in the Natural Language Processing (NLP) domain and has achieved promising results in identifying workflow-descriptive paragraphs, generating workflow phrases, and categorizing these phrases into data preparation, data processing, and data analysis stages.

Key Takeaways:

  • The researchers propose an end-to-end framework for automatically generating comprehensive research workflows from full-text academic papers.
  • The framework utilizes Positive-Unlabeled (PU) Learning with SciBERT to identify workflow-descriptive paragraphs, achieving an F1-score of 0.9772.
  • The framework utilizes Flan-T5 with prompt learning to generate workflow phrases from these paragraphs, yielding ROUGE-1, ROUGE-2, and ROUGE-L scores of 0.4543, 0.2877, and 0.4427, respectively.
  • The framework uses ChatGPT with few-shot learning to categorize phrases into data preparation, data processing, and data analysis stages, achieving a classification precision of 0.958.
  • The framework generates readable visual flowcharts of the entire research workflows.
  • The research reveals key methodological shifts over the past two decades in the NLP domain, including the increasing emphasis on data analysis and the transition from feature engineering to ablation studies.

Statistics:

  • The F1-score of the framework in identifying workflow-descriptive paragraphs is 0.9772.
  • The ROUGE-1, ROUGE-2, and ROUGE-L scores of the framework in generating workflow phrases are 0.4543, 0.2877, and 0.4427, respectively.
  • The classification precision of the framework in categorizing phrases into data preparation, data processing, and data analysis stages is 0.958.

Sources:

  • Research paper: Automated Generation of Research Workflows From Academic Papers: a Full-text Mining Framework. Journal of Informetrics, 2025;19(4).
  • Source code and data: available at: https://github.com/ZH-heng/research_workflow.
  • Journal of Informetrics: www.journals.elsevier.com/journal-of-informetrics.
  • Institute contact: Chengzhi Zhang, Nanjing University of Science and Technology, Dept. of Information Management, Nanjing 210094, People's Republic of China.
  • National Natural Science Foundation of China (NSFC).
  • Note: The source paper is published in Journal of Informetrics, vol. 19, issue 4, in 2025.

Read more