Pixel-Perfect Structure-From-Motion With Featuremetric Refinement

Finding local features that are repeatable across multiple views is a cornerstone of sparse 3D reconstruction. The classical image matching paradigm detects keypoints per-image once and for all, which can yield poorly-localized features and propagate large errors to the final geometry. In this artic...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence Vol. 47; no. 5; pp. 3298 - 3309
Main Authors: Sarlin, Paul-Edouard, Lindenberger, Philipp, Larsson, Viktor, Pollefeys, Marc
Format: Journal Article
Language:English
Published: United States IEEE 01.05.2025
Subjects:
Bundle adjustment
Computer and Information Sciences
Computer graphics and computer vision
Costs
Data- och informationsvetenskap (Datateknik)
Datorgrafik och datorseende
Estimation
Feature extraction
feature matching
featuremetric optimization
Geometry
Image reconstruction
Location awareness
Natural Sciences
Naturvetenskap
Optimization
structure-from-Motion
visual localization
ISSN:0162-8828, 1939-3539, 2160-9292, 1939-3539
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Finding local features that are repeatable across multiple views is a cornerstone of sparse 3D reconstruction. The classical image matching paradigm detects keypoints per-image once and for all, which can yield poorly-localized features and propagate large errors to the final geometry. In this article, we refine two key steps of structure-from-motion by a direct alignment of low-level image information from multiple views: we first adjust the initial keypoint locations prior to any geometric estimation, and subsequently refine points and camera poses as a post-processing. This refinement is robust to large detection noise and appearance changes, as it optimizes a featuremetric error based on dense features predicted by a neural network. This significantly improves the accuracy of camera poses and scene geometry for a wide range of keypoint detectors, challenging viewing conditions, and off-the-shelf deep features. Our system easily scales to large image collections, enabling pixel-perfect crowd-sourced localization at scale.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0162-8828
1939-3539
2160-9292
1939-3539
DOI:10.1109/TPAMI.2023.3237269