LGT-Net: Indoor Panoramic Room Layout Estimation with Geometry-Aware Transformer Network

3D room layout estimation by a single panorama using deep neural networks has made great progress. However, previous approaches can not obtain efficient geometry awareness of room layout with the only latitude of boundaries or horizon-depth. We present that using horizondepth along with room height...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 1644 - 1653
Main Authors: Jiang, Zhigang, Xiang, Zhongzheng, Xu, Jinhua, Zhao, Ming
Format: Conference Proceeding
Language:English
Published: IEEE 01.06.2022
Subjects:
3D from single images; Datasets and evaluation; Deep learning architectures and techniques; Photogrammetry and remote sensing; Recognition: detection
categorization
Deep learning
Estimation
Geometry
grouping and shape analysis; Vision applications and systems
Layout
Neural networks
retrieval; Robot vision; Scene analysis and understanding; Segmentation
Solid modeling
Three-dimensional displays
ISSN:1063-6919
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:3D room layout estimation by a single panorama using deep neural networks has made great progress. However, previous approaches can not obtain efficient geometry awareness of room layout with the only latitude of boundaries or horizon-depth. We present that using horizondepth along with room height can obtain omnidirectionalgeometry awareness of room layout in both horizontal and vertical directions. In addition, we propose a planar-geometry aware loss function with normals and gradients of normals to supervise the planeness of walls and turning of corners. We propose an efficient network, LGT-Net, for room layout estimation, which contains a novel Transformer architecture called SWG-Transformer to model geometry relations. SWG-Transformer consists of (Shifted) Window Blocks and Global Blocks to combine the local and global geometry relations. Moreover, we design a novel relative position embedding of Transformer to enhance the spatial identification ability for the panorama. Experiments show that the proposed LGT-Net achieves better performance than current state-of-the-arts (SOTA) on benchmark datasets. The code is publicly available at https://github.com/zhigangjiang/LGT-Net.
ISSN:1063-6919
DOI:10.1109/CVPR52688.2022.00170