MPR-GAN: A Novel Neural Rendering Framework for MLS Point Cloud with Deep Generative Learning

Efficient point cloud visualization is indispensable for practical applications. In the context of point cloud visualization, 3D rendering can be viewed as the kernel that transforms 3D points into a 2D scene image. Compared with traditional point-based rendering (PBR), neural image-based rendering...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on geoscience and remote sensing Vol. 60; p. 1
Main Authors: Xu, Qingyang, Guan, Xuefeng, Cao, Jun, Ma, Yanli, Wu, Huayi
Format: Journal Article
Language:English
Published: New York IEEE 2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
CGAN
Coders
colorization
Context
Frameworks
Generative adversarial networks
Image quality
Lasers
MLS point cloud
neural image-based rendering
Rendering
Scaling
Signal to noise ratio
Similarity
Three dimensional models
transformation
Visualization
ISSN:0196-2892, 1558-0644
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Efficient point cloud visualization is indispensable for practical applications. In the context of point cloud visualization, 3D rendering can be viewed as the kernel that transforms 3D points into a 2D scene image. Compared with traditional point-based rendering (PBR), neural image-based rendering (NIBR) has gradually emerged as a feasible solution for point cloud rendering. To efficiently render sparse and colorless MLS point cloud, we propose a novel neural rendering framework based on deep generative learning, named MPR-GAN. In this framework, perspective projection with intrinsic parameter scaling and cumulative distribution normalization is first utilized to transform the 3D point cloud into a compact 2D image; a CGAN-based rendering model is then proposed to generate a photorealistic scene image from the projected 2D image. In this CGAN model, the asymmetric encoder-decoder generator can implement inpainting and true colorization using context feature capturing and edge information perception; a multi-scale discriminator is built to guarantee the model output with global consistency and local details. Moreover, a hybrid loss function is designed to improve the visual quality of the generated images with similarity constraints from both the content and structure. Two public MLS point cloud datasets are selected and employed to carry out extensive evaluation using MPR-GAN and other baseline frameworks. The experimental results demonstrate that MPR-GAN achieves the state-of-the-art rendering performance in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). Furthermore, the efficiency analysis shows that MPR-GAN can support real-time rendering, achieving end-to-end rendering from raw points.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0196-2892
1558-0644
DOI:10.1109/TGRS.2022.3212389