Content Adaptive Neural Network Based In-Loop Filter in VVC Using Dual Query Transformer

Strategic Insights -

Researchers from Xidian University in Xi'an, People's Republic of China, have made a significant breakthrough in the field of neural networks for video compression. By proposing a novel content adaptive neural network-based in-loop filter (NNLF) using a dual query transformer (DQT), they have achieved impressive results in reducing compression artifacts and improving video quality.

Key Takeaways:

  • The DQT-CALF model is a content adaptive NNLF in the versatile video coding (VVC) framework that uses dual query transformer to effectively fuse low-frequency global features and high-frequency local features.
  • The model consists of three parts: feature extraction, feature enhancement, and reconstruction, where the feature extraction part uses a parallel network structure for the main and auxiliary inputs and assigns feature maps to different weights according to the richness of the input information.
  • The feature enhancement part uses a multi-type feature fusion module that divides the input tensor into low-frequency and high-frequency features from the frequency viewpoint and local and global features from the spatial viewpoint.
  • The model achieves average BD rate gains of {8.79% (Y), 22.09% (U), 22.99% (V)} and {9.68% (Y), 22.27% (U), 22.52% (V)} over the VTM-11.0_NNVC-3.0 anchor under all intra (AI) and random access (RA) configurations, respectively.
  • The research has been peer-reviewed and published in the journal Neurocomputing.
  • The study has been funded by the National Natural Science Foundation of China (NSFC).
  • The research team, led by Cheolkon Jung from Xidian University, has developed a novel model that effectively fuses low-frequency and high-frequency features and reduces the risk of information loss.

Statistics:

  • Results show an average BD rate gain of 8.79% (Y), 22.09% (U), and 22.99% (V) and 9.68% (Y), 22.27% (U), and 22.52% (V) under all intra (AI) and random access (RA) configurations, respectively.
  • The model achieves significant improvements in compression artifacts and video quality.
  • The study has been published in the journal Neurocomputing, volume 637.
  • The research has been funded by the National Natural Science Foundation of China (NSFC).

Sources:

  • NewsRx. Data on Networks Discussed by Researchers at Xidian University (Dqt-calf: Content Adaptive Neural Network Based In-loop Filter In Vvc Using Dual Query Transformer). Journal of Engineering. July 7, 2025; p 563.
  • Neurocomputing. Dqt-calf: Content Adaptive Neural Network Based In-loop Filter In Vvc Using Dual Query Transformer. Volume 637.