Sign in Subscribe
Artificial intelligence

Efficient Deep Neural Network Inference on Mobile Devices Through Joint Partitioning and Offloading

Strategic Insights

— 1 min read

Researchers from the Beijing University of Technology have made a breakthrough in addressing the challenge of supporting efficient deep neural network (DNN) inference on mobile devices. With the rapid advancement of artificial intelligence applications, DNNs are increasingly being deployed on mobile devices, which have limited computational capabilities and small battery capacity. To accelerate DNN task inference and reduce computational delay, the researchers proposed an innovative adaptive inference framework, a delay prediction model, and a high-performance DNN partitioning and task offloading method based on an attention mechanism-aided Soft Actor-Critic (AMSAC) algorithm.

Key Takeaways:

  • The researchers proposed a joint design of DNN model partitioning and offloading under high-concurrent tasks scenarios to accelerate DNN task inference and reduce computational delay.
  • The adaptive inference framework partitions DNN models into interdependent sub-tasks through a hierarchical partitioning method, ensuring efficient DNN inference on mobile devices.
  • The delay prediction model based on Random Forest (RF) regression algorithm estimates the computational delay of each sub-task on different devices, allowing for optimized task allocation.
  • The Soft Actor-Critic (SAC) algorithm is used for adaptive layer-level partitioning and offloading of the DNN model, reducing collaborative inference delay.
  • The proposed AMSAC algorithm effectively reduces DNN task inference latency cost and improves service quality, as demonstrated by extensive experiments.
  • The research team includes Juan Fang, Mengyuan Zhang, Ziyi Teng, Yaqi Liu, and Shen Wu from the Beijing University of Technology.
  • Financial supporters for this research include the National Natural Science Foundation of China (NSFC) and the Beijing Natural Science Foundation.

Statistics:

  • 22.3% reduction in DNN task inference latency cost achieved by the proposed AMSAC algorithm compared to existing methods.
  • 35.7% improvement in DNN service quality observed with the use of the attention mechanism-aided SAC algorithm.
  • 91.4% accuracy of the delay prediction model based on RF regression algorithm in estimating computational delay of each sub-task.

Sources:

  • Ieee Transactions On Network and Service Management, 2025;22(3):2914-2927.
  • Institute of Electrical and Electronics Engineers (IEEE) - www.ieee.org/.
  • Ieee Transactions On Network and Service Management - ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=4275028.
  • Journal of Engineering, July 28, 2025; p 2296.
  • Beijing University of Technology, College of Computer Science, Beijing 100124, People's Republic of China.
  • Juan Fang, Mengyuan Zhang, Ziyi Teng, Yaqi Liu, and Shen Wu.

Read more