LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation
Saved in:
| Title: | LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation |
|---|---|
| Authors: | Jiacheng Li, Chang Chen, Fenglong Song, Youliang Yan, Zhiwei Xiong |
| Source: | IEEE Transactions on Pattern Analysis and Machine Intelligence. 47:8453-8470 |
| Publication Status: | Preprint |
| Publisher Information: | Institute of Electrical and Electronics Engineers (IEEE), 2025. |
| Publication Year: | 2025 |
| Subject Terms: | FOS: Computer and information sciences, Computer Vision and Pattern Recognition (cs.CV), Image and Video Processing (eess.IV), Computer Science - Computer Vision and Pattern Recognition, FOS: Electrical engineering, electronic engineering, information engineering, Electrical Engineering and Systems Science - Image and Video Processing, Computer Science - Multimedia, Multimedia (cs.MM) |
| Description: | Image resampling is a basic technique that is widely employed in daily applications, such as camera photo editing. Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors. Still, these methods are not the perfect substitute for interpolation, due to the drawbacks in efficiency and versatility. In this work, we propose a novel method of Learning Resampling Function (termed LeRF), which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption of interpolation. Specifically, LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the hyper-parameters that determine the shapes of these resampling functions with a neural network. Based on the formulation of LeRF, we develop a family of models, including both efficiency-orientated and performance-orientated ones. To achieve interpolation-level efficiency, we adopt look-up tables (LUTs) to accelerate the inference of the learned neural network. Furthermore, we design a directional ensemble strategy and edge-sensitive indexing patterns to better capture local structures. On the other hand, to obtain DNN-level performance, we propose an extension of LeRF to enable it in cooperation with pre-trained upsampling models for cascaded resampling. Extensive experiments show that the efficiency-orientated version of LeRF runs as fast as interpolation, generalizes well to arbitrary transformations, and outperforms interpolation significantly, e.g., up to 3dB PSNR gain over Bicubic for x2 upsampling on Manga109. Besides, the performance-orientated version of LeRF reaches comparable performance with existing DNNs at much higher efficiency, e.g., less than 25% running time on a desktop GPU. Code: https://github.com/ddlee-cn/LeRF-PyTorch |
| Document Type: | Article |
| ISSN: | 1939-3539 0162-8828 |
| DOI: | 10.1109/tpami.2025.3577227 |
| DOI: | 10.48550/arxiv.2407.09935 |
| Access URL: | http://arxiv.org/abs/2407.09935 |
| Rights: | IEEE Copyright CC BY |
| Accession Number: | edsair.doi.dedup.....1d1ee27e7f2f355b20825f9b8c558be7 |
| Database: | OpenAIRE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstract: | Image resampling is a basic technique that is widely employed in daily applications, such as camera photo editing. Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors. Still, these methods are not the perfect substitute for interpolation, due to the drawbacks in efficiency and versatility. In this work, we propose a novel method of Learning Resampling Function (termed LeRF), which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption of interpolation. Specifically, LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the hyper-parameters that determine the shapes of these resampling functions with a neural network. Based on the formulation of LeRF, we develop a family of models, including both efficiency-orientated and performance-orientated ones. To achieve interpolation-level efficiency, we adopt look-up tables (LUTs) to accelerate the inference of the learned neural network. Furthermore, we design a directional ensemble strategy and edge-sensitive indexing patterns to better capture local structures. On the other hand, to obtain DNN-level performance, we propose an extension of LeRF to enable it in cooperation with pre-trained upsampling models for cascaded resampling. Extensive experiments show that the efficiency-orientated version of LeRF runs as fast as interpolation, generalizes well to arbitrary transformations, and outperforms interpolation significantly, e.g., up to 3dB PSNR gain over Bicubic for x2 upsampling on Manga109. Besides, the performance-orientated version of LeRF reaches comparable performance with existing DNNs at much higher efficiency, e.g., less than 25% running time on a desktop GPU.<br />Code: https://github.com/ddlee-cn/LeRF-PyTorch |
|---|---|
| ISSN: | 19393539 01628828 |
| DOI: | 10.1109/tpami.2025.3577227 |