SA3D-L: A lightweight model for 3D object segmentation using neural radiance fields

The Segment Anything Model (SAM) has recently made significant progress in object segmentation within 2D images. However, the task of segmenting objects in a 3D space remains a primary hurdle in computer vision. The neural radiance field (NeRF) utilizes a multilayer perceptron (MLP) to effectively l...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	Neurocomputing (Amsterdam) Ročník 623; s. 129420
Hlavní autoři:	Liu, Jian, Yu, Zhen
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Elsevier B.V 28.03.2025
Témata:	2D–3D segmentation Multi-layer perceptron (MLP) Neural radiation field (NeRF) Novel view synthesis Segment Anything Model (SAM) 2D–3D segmentation Multi-layer perceptron (MLP) Neural radiation field (NeRF) Segment Anything Model (SAM) Novel view synthesis
ISSN:	0925-2312
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	The Segment Anything Model (SAM) has recently made significant progress in object segmentation within 2D images. However, the task of segmenting objects in a 3D space remains a primary hurdle in computer vision. The neural radiance field (NeRF) utilizes a multilayer perceptron (MLP) to effectively learn the continuous representation of a 3D scene. Due to its consistent 3D perspectives from various angles, SAM, originally designed for 2D segmentation, can be extended for 3D object segmentation by incorporating NeRF. However, a limitation of NeRF is that the MLP encapsulates the whole scene as a single representation, without distinguishing individual objects. This study introduces a lightweight 3D Segment Anything Model (SA3D-L), which separately represents each segmented object within a scene by modifying the MLP output. Experimental results on established benchmarks revealed that the 3D segmentation representations of objects can be derived from their 2D masks, allowing the independent manipulation of segmented objects and the reconstruction of a new scene. The code is available at: https://github.com/liujian0819/SA3D-L. •We proposed a method to learn the representation of 3D segmentation from 2D masks.•We utilized a single MLP to represent multiple objects in the scene simultaneously.•We can separately manipulate each segmented object and reconstruct a new scene.
ISSN:	0925-2312
DOI:	10.1016/j.neucom.2025.129420