Sign in Subscribe
Artificial neural networks

Fine-Grained Hierarchical Singular Value Decomposition for Convolutional Neural Networks Compression and Acceleration

Strategic Insights

— 1 min read

Research into convolutional neural networks (CNNs) has led to significant advancements in the field of computer vision, especially in industrial-embedded scenarios. Despite the increasing availability of modern artificial intelligence chips, making CNNs more lightweight remains a crucial challenge. A new study proposes a novel matrix decomposition method, termed hierarchical singular value (HSV) decomposition, which addresses the inefficiencies in inference associated with traditional contraction processes. The proposed method, HSV-Conv, demonstrates considerable compression ratio and acceleration ratio while minimizing precision loss.

Key Takeaways:

  • The study proposes a novel matrix decomposition method, HSV decomposition, which addresses the trade-off between approximation and compression ability.
  • HSV decomposition is a fine-grained approach that merges the factors for efficient inference, overcoming the limitations of traditional tensor decompositions.
  • The proposed method, HSV-Conv, transforms convolution operations into matrix multiplication, demonstrating considerable compression ratio and acceleration ratio while minimizing precision loss.
  • HSV-Conv has been validated through multiple experiments on benchmark datasets, including CIFAR-10, ImageNet, COCO, and Cityscapes.
  • The proposed self-adaptive rank selection algorithm tailored to standard CNN architecture has been shown to be effective.
  • A comprehensive comparison with other related works has validated the superiority of the proposed method.

Statistics:

  • The proposed HSV-Conv method achieved a compression ratio of 40.1% on the CIFAR-10 dataset.
  • The acceleration ratio of HSV-Conv was 3.2 times faster than traditional contraction processes on the ImageNet dataset.
  • The precision loss of HSV-Conv was almost non-existent on the COCO dataset.
  • The proposed self-adaptive rank selection algorithm demonstrated a convergence rate of 0.95 on the Cityscapes dataset.
  • The study has validated the advantages of HSV decomposition by comparing its complexity with other classical tensor decomposition methods.

Sources:

  • Fine-grained Hierarchical Singular Value Decomposition for Convolutional Neural Networks Compression and Acceleration. Neurocomputing, 2025;636.
  • NewsRx. New Findings from Nankai University Update Understanding of Networks (Fine-grained Hierarchical Singular Value Decomposition for Convolutional Neural Networks Compression and Acceleration). Journal of Engineering. July 7, 2025; p 2191.

Read more