Fusion of Images of Different Spectra Based on Generative Adversarial Networks - Vizil’ter, Yu. V.; Vygolov, O. V.; Komarov, D. V.; Lebedev, M. A. (Journal of computer & systems sciences international, 2019)
A method for fusing images of different spectra by using generative adversarial networks is proposed. An original architecture of a FusionNet neural network is developed based on pix2pix. It enables the synthesis of a complex (integrated) image that comprises the most informative fragments of different-spectra images, thus being more informative than any of these individual images. A technique for generating training and test sets, as well as the process of data augmentation, is described. The operation of the proposed image fusion method is demonstrated on some real-world infrared and visible images.