Over-the-Air Learning-based Geometry Point Cloud Transmission

3D point cloud is a three-dimensional data format generated by LiDARs and depth sensors, and is being increasingly used in a large variety of applications from autonomous vehicles to robotics and metaverse. This paper presents novel solutions for the efficient and reliable transmission of point clou...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE journal on selected areas in communications s. 1
Hlavní autori: Bian, Chenghong, Shao, Yulin, Gunduz, Deniz
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: IEEE 2025
Predmet:
Decoding
Deep joint source channel coding (DeepJSCC)
Feature extraction
Image reconstruction
Neural networks
Neural radiance field
neural radiance field (NeRF)
over-the-air
point cloud
Point cloud compression
Three-dimensional displays
Transformers
Vectors
Wireless communication
ISSN:0733-8716, 1558-0008
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:3D point cloud is a three-dimensional data format generated by LiDARs and depth sensors, and is being increasingly used in a large variety of applications from autonomous vehicles to robotics and metaverse. This paper presents novel solutions for the efficient and reliable transmission of point clouds over wireless channels for real-time applications. We first propose SEmatic Point cloud Transmission (SEPT) for small-scale point clouds, which encodes the point cloud via an iterative downsampling and feature extraction process. At the receiver, SEPT decoder reconstructs the point cloud with latent reconstruction and offset-based upsampling. A novel channel-adaptive module is proposed to allow SEPT to operate effectively over a wide range of channel conditions. Next, we propose OTA-NeRF, a scheme inspired by neural radiance fields. OTA-NeRF performs voxelization to the point cloud input and learns to encode the voxelized point cloud into a neural network. Instead of transmitting the extracted feature vectors as in SEPT, it transmits the learned neural network weights over the air in an analog fashion along with few hyperparameters that are transmitted digitally. At the receiver, the OTA-NeRF decoder reconstructs the original point cloud using the received noisy neural network weights. To further increase the bandwidth efficiency of the OTA-NeRF scheme, a fine-tuning algorithm is developed, where only a fraction of the neural network weights are retrained and transmitted. Noticing the poor generality of the OTA-NeRF schemes where the neural network weights are trained for a specific point cloud, we propose an alternative approach, termed OTA-MetaNeRF, which encodes different input point clouds into the latent vectors with shared neural network weights. Extensive numerical experiments confirm that the proposed SEPT, OTA-NeRF and OTA-MetaNeRF schemes achieve superior or comparable performance over the conventional approaches, where an octree-based or a learning-based point cloud compression scheme is concatenated with a channel code. As an additional advantage, all schemes mitigate the cliff and leveling effects making them particularly attractive for highly mobile scenarios. Finally, the run-time complexities of the schemes are evaluated to verify the capability of the proposed schemes for real-time communications.
ISSN:0733-8716
1558-0008
DOI:10.1109/JSAC.2025.3623164