Dual-hierarchical knowledge distillation for video captioning - Luo, HuiLan; Wan, SiQi; Cai, Xia; Wang, ChanJuan; Chen, HongKun (Pattern recognition, 2026)
•Introduce a dual-hierarchical distillation framework for video captioning.•Propose a caption quality grading method to reduce annotation noise.•Distill semantic features into a lightweight student model for efficiency.•Improve captioning accuracy while lowering computational cost.•Outperform prior methods on MSVD and MSR-VTT benchmarks. Video captioning remains a challenging task due to the tradeoff between accuracy and computational efficiency. We propose CapDistill, a dual hierarchical distillation framework that transfers semantic knowledge from a powerful teacher model to a lightweight student model. CapDistill captures object-level and action-level semantics from captions and transfers multilevel knowledge including object features, action features, and word-level predictions through a hierarchical strategy. To reduce the impact of noisy annotations, we introduce a caption quality grading mechanism that assigns quality-based weights to training captions. Experiments on MSR-VTT and MSVD demonstrate that CapDistill achieves state-of-the-art accuracy while significantly reducing inference cost. Code is available at: https://github.com/ccc000-png/SFTCap.