Advancing Arbitrary-Scale Image Super-Resolution: Introducing Residual In Residual Dense Networks with a Novel Local Implicit Image Function
Abstract
The progress in image super-resolution has seen significant advancements due to the emergence of deep convolutional neural networks. While most researchers concentrate on training models for specific scales, only a few delve into creating models adaptable to various scales. This study builds upon prior research that focuses on achieving arbitrary resolution with a single model. The model under consideration employs an auto-encoder structure. The encoder extracts feature maps from the input image, while the decoder reconstructs these feature maps to the resolution specified by the user. Referred to as RRDN-NLIIF (Residual in Residual Dense Networks with Novel Local Implicit Image Function), our experimental results demonstrate its superior performance over the benchmark model in terms of the PSNR metric.
References
X. Hu, H. Mu, X. Zhang, Z. Wang, T. Tan, and J. Sun, "Meta-SR: A magnifica-tion-arbitrary network for super-resolution," in Proceedings of the IEEE/CVF con-ference on computer vision and pattern recognition, pp. 1575-1584. 2019.
Y. Chen, S. Liu, and X. Wang, "Learning continuous image representation with local implicit image function," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8628-8638. 2021.
J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove, "Deepsdf: Learning continuous signed distance functions for shape representation," in Pro-ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 165-174. 2019.
L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger, "Occupancy networks: Learning 3d reconstruction in function space," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4460-4470. 2019.
Z. Chen and H. Zhang, "Learning implicit fields for generative shape modeling," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog-nition, pp. 5939-5948. 2019.
S. Saito, Z. Huang, R. Natsume, S. Morishima, A. Kanazawa, and H. Li, "Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2304-2314. 2019.
C. Jiang, A. Sud, A. Makadia, J. Huang, M. Nießner, and T. Funkhouser, "Local implicit grid representations for 3d scenes," in Proceedings of the IEEE/CVF Con-ference on Computer Vision and Pattern Recognition, pp. 6001-6010. 2020.
V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein, "Implicit neural representations with periodic activation functions," Advances in Neural Information Processing Systems, vol. 33, pp. 7462-7473, 2020.
X. Wang et al., "Esrgan: Enhanced super-resolution generative adversarial net-works," in Proceedings of the European conference on computer vision (ECCV) workshops, pp. 0-0. 2018.
X. Xu, Z. Wang, and H. Shi, "Ultrasr: Spatial encoding is a missing key for implicit image function-based arbitrary-scale super-resolution," arXiv preprint arXiv:2103.12716, 2021.
C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE transactions on pattern analysis and machine intel-ligence, vol. 38, no. 2, pp. 295-307, 2015.
J. Kim, J. K. Lee, and K. M. Lee, "Accurate image super-resolution using very deep convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1646-1654. 2016.
J. Kim, J. K. Lee, and K. M. Lee, "Deeply-recursive convolutional network for image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1637-1645. 2016.
C. Ledig et al., "Photo-realistic single image super-resolution using a generative adversarial network," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681-4690. 2017.
Y. Blau, R. Mechrez, R. Timofte, T. Michaeli, and L. Zelnik-Manor, "The 2018 PIRM challenge on perceptual image super-resolution," in Proceedings of the Eu-ropean Conference on Computer Vision (ECCV) Workshops, pp. 0-0. 2018.
B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, "Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 136-144. 2017.
Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2472-2481. 2018.
A. Vaswani et al., "Attention is all you need," Advances in neural information processing systems, vol. 30, 2017.
B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, "Nerf: Representing scenes as neural radiance fields for view synthesis," Communications of the ACM, vol. 65, no. 1, pp. 99-106, 2021.
L. Liu, J. Gu, K. Zaw Lin, T.-S. Chua, and C. Theobalt, "Neural sparse voxel fields," Advances in Neural Information Processing Systems, vol. 33, pp. 15651-15663, 2020.
K. Zhang, G. Riegler, N. Snavely, and V. Koltun, "Nerf++: Analyzing and im-proving neural radiance fields," arXiv preprint arXiv:2010.07492, 2020.
K. Schwarz, Y. Liao, M. Niemeyer, and A. Geiger, "Graf: Generative radiance fields for 3d-aware image synthesis," Advances in Neural Information Processing Systems, vol. 33, pp. 20154-20166, 2020.
S. Wizadwongsa, P. Phongthawee, J. Yenphraphai, and S. Suwajanakorn, "Nex: Real-time view synthesis with neural basis expansion," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8534-8543. 2021.
E. Agustsson and R. Timofte, "NTIRE 2017 Challenge on Single Image Su-per-Resolution: Dataset and Study," in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1122-1131, doi: 10.1109/CVPRW.2017.150. 21-26 July 2017 2017.
M. Bevilacqua, A. Roumy, C. Guillemot, and M.-L. Alberi Morel, "Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding," in British Machine Vision Conference (BMVC), Guildford, Surrey, United Kingdom, https://hal.inria.fr/hal-00747054/document
https://hal.inria.fr/hal-00747054/file/bmvc_final_submitted.pdf. [Online]. Availa-ble: https://hal.inria.fr/hal-00747054. [Online]. Available: https://hal.inria.fr/hal-00747054, 2012-09 2012.
R. Zeyde, M. Elad, and M. Protter, On Single Image Scale-Up Using Sparse-Representations. 2010, pp. 711-730.
D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proceedings Eighth IEEE International Confer-ence on Computer Vision. ICCV 2001, vol. 2, pp. 416-423 vol.2, doi: 10.1109/ICCV.2001.937655. 7-14 July 2001 2001.
J. B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from trans-formed self-exemplars," in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5197-5206, doi: 10.1109/CVPR.2015.7299156. 7-12 June 2015 2015.
T. Karras, T. Aila, S. Laine, and J. Lehtinen, "Progressive growing of gans for improved quality, stability, and variation," arXiv preprint arXiv:1710.10196, 2017.
Z. Liu, P. Luo, X. Wang, and X. Tang, "Deep learning face attributes in the wild," in Proceedings of the IEEE international conference on computer vision, pp. 3730-3738. 2015.
X. Chen, X. Wang, J. Zhou, and C. Dong, "Activating More Pixels in Image Su-per-Resolution Transformer," arXiv preprint arXiv:2205.04437, 2022.
Y. Chen and Y. Wu, "Arbitrary-Scale Image Super Resolution Using Residual in Residual Dense Networks with New Local Implicit Image Function," in Proceedings of the14th IIAI International Congress on Advanced Applied Informatics, 2023.
