DSAC - Differentiable RANSAC for Camera Localization

RANSAC is an important algorithm in robust optimization and a central building block for many computer vision applications. In recent years, traditionally hand-crafted pipelines have been replaced by deep learning pipelines, which can be trained in an end-to-end fashion. However, RANSAC has so far n...

Full description

Saved in:
Bibliographic Details
Published in:2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2492 - 2500
Main Authors: Brachmann, Eric, Krull, Alexander, Nowozin, Sebastian, Shotton, Jamie, Michel, Frank, Gumhold, Stefan, Rother, Carsten
Format: Conference Proceeding
Language:English
Published: IEEE 01.07.2017
Subjects:
Cameras
Computer vision
Machine learning
Pipelines
Robustness
Three-dimensional displays
ISSN:1063-6919, 1063-6919
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract RANSAC is an important algorithm in robust optimization and a central building block for many computer vision applications. In recent years, traditionally hand-crafted pipelines have been replaced by deep learning pipelines, which can be trained in an end-to-end fashion. However, RANSAC has so far not been used as part of such deep learning pipelines, because its hypothesis selection procedure is non-differentiable. In this work, we present two different ways to overcome this limitation. The most promising approach is inspired by reinforcement learning, namely to replace the deterministic hypothesis selection by a probabilistic selection for which we can derive the expected loss w.r.t. to all learnable parameters. We call this approach DSAC, the differentiable counterpart of RANSAC. We apply DSAC to the problem of camera localization, where deep learning has so far failed to improve on traditional approaches. We demonstrate that by directly minimizing the expected loss of the output camera poses, robustly estimated by RANSAC, we achieve an increase in accuracy. In the future, any deep learning pipeline can use DSAC as a robust optimization component.
AbstractList RANSAC is an important algorithm in robust optimization and a central building block for many computer vision applications. In recent years, traditionally hand-crafted pipelines have been replaced by deep learning pipelines, which can be trained in an end-to-end fashion. However, RANSAC has so far not been used as part of such deep learning pipelines, because its hypothesis selection procedure is non-differentiable. In this work, we present two different ways to overcome this limitation. The most promising approach is inspired by reinforcement learning, namely to replace the deterministic hypothesis selection by a probabilistic selection for which we can derive the expected loss w.r.t. to all learnable parameters. We call this approach DSAC, the differentiable counterpart of RANSAC. We apply DSAC to the problem of camera localization, where deep learning has so far failed to improve on traditional approaches. We demonstrate that by directly minimizing the expected loss of the output camera poses, robustly estimated by RANSAC, we achieve an increase in accuracy. In the future, any deep learning pipeline can use DSAC as a robust optimization component.
Author Brachmann, Eric
Rother, Carsten
Shotton, Jamie
Krull, Alexander
Nowozin, Sebastian
Gumhold, Stefan
Michel, Frank
Author_xml – sequence: 1
  givenname: Eric
  surname: Brachmann
  fullname: Brachmann, Eric
  organization: Tech. Univ. Dresden, Dresden, Germany
– sequence: 2
  givenname: Alexander
  surname: Krull
  fullname: Krull, Alexander
  organization: Tech. Univ. Dresden, Dresden, Germany
– sequence: 3
  givenname: Sebastian
  surname: Nowozin
  fullname: Nowozin, Sebastian
– sequence: 4
  givenname: Jamie
  surname: Shotton
  fullname: Shotton, Jamie
– sequence: 5
  givenname: Frank
  surname: Michel
  fullname: Michel, Frank
  organization: Tech. Univ. Dresden, Dresden, Germany
– sequence: 6
  givenname: Stefan
  surname: Gumhold
  fullname: Gumhold, Stefan
  organization: Tech. Univ. Dresden, Dresden, Germany
– sequence: 7
  givenname: Carsten
  surname: Rother
  fullname: Rother, Carsten
  organization: Tech. Univ. Dresden, Dresden, Germany
BookMark eNpNjLtOw0AQAA8UJOJASUXjH3DYvb2Ht7QcXpIFKDza6OyspUOOjWw38PUIQUE1oykmUYt-6EWpC4Q1IvBV-fa0XWtAv9bOH6kELeUOjPXmWC0RHGWOkRf__FQl0_QOoMlrWCqzeS7KNEs3sW1llH6Ooe4k3RYPP70dxrQMBxlDWg1N6OJXmOPQn6mTNnSTnP9xpV5vrl_Ku6x6vL0viyqL6O2cNU7XlnDvLTtwIK4hJ8hg9izMDQpJrjU2rQnaAgYmT85TbQghgFhaqcvfbxSR3ccYD2H83OXA7C3QN7WDREA
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/CVPR.2017.267
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Applied Sciences
Computer Science
EISBN 1538604574
9781538604571
EISSN 1063-6919
EndPage 2500
ExternalDocumentID 8099750
Genre orig-research
GroupedDBID 23M
29F
29O
6IE
6IH
6IK
ABDPE
ACGFS
ALMA_UNASSIGNED_HOLDINGS
CBEJK
IPLJI
M43
RIE
RIO
RNS
ID FETCH-LOGICAL-i175t-c62b531d7596060e6c36e1904d9e99c1e3e8221cf4a2501a9373673b4310a0e53
IEDL.DBID RIE
ISICitedReferencesCount 398
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000418371402059&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1063-6919
IngestDate Wed Aug 27 02:33:39 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i175t-c62b531d7596060e6c36e1904d9e99c1e3e8221cf4a2501a9373673b4310a0e53
PageCount 9
ParticipantIDs ieee_primary_8099750
PublicationCentury 2000
PublicationDate 2017-July
PublicationDateYYYYMMDD 2017-07-01
PublicationDate_xml – month: 07
  year: 2017
  text: 2017-July
PublicationDecade 2010
PublicationTitle 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
PublicationTitleAbbrev CVPR
PublicationYear 2017
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0023720
ssj0003211698
Score 2.5941386
Snippet RANSAC is an important algorithm in robust optimization and a central building block for many computer vision applications. In recent years, traditionally...
SourceID ieee
SourceType Publisher
StartPage 2492
SubjectTerms Cameras
Computer vision
Machine learning
Pipelines
Robustness
Three-dimensional displays
Title DSAC - Differentiable RANSAC for Camera Localization
URI https://ieeexplore.ieee.org/document/8099750
WOSCitedRecordID wos000418371402059&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LawIxEA5WeujJtlr6JoceG93dZPM4ylrpQURsK94kyY4gFC0--vs7WdeVQi-9hTmFSYb55vHNEPKU5NZJL3P8vKCY8BqYhlQxqyDiRhtpi2HVk4EaDvV0akY18lxxYQCgaD6DdjgWtfx85XchVdbRgeYZAvQTpdSeq1XlUzhGMtJUFYQkbF8pKp2SM2lic5yv2ckmo3Fo6lLtRP7eqlI4lX7jf9c5J60jO4-OKr9zQWqwvCSNEk7S0lg3KDpsbDjImkT03roZZbRXrkVB83afQMfdYZAjfqWZDVkqOggurqRotshH_-U9e2Xl3gS2QDCwZV4mDk0rV2kITyKQnktAxy9yA8b4GDggLIj9XFgEQLFFhMKl4g6xRGQjSPkVqS9XS7gmNJFeKHxAb7QTqbDOee0cx7ByrtJc5jekGfQy-9qPxpiVKrn9W3xHzoLa992u96S-Xe_ggZz67-1is34s3vMHIfSbgg
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEG4ImugJFYxve_BoYXfbbbdHskgwrhuCSLiRtjskJAQMD3-_7bIsMfHirZlT0-lkvnl8Mwg9BZnS3PDMfl4QhJkISAShIEqAR2UkucqHVY8SkabReCz7FfRccmEAIG8-g6Y75rX8bGm2LlXWihzN0wXoRyFjgb9ja5UZFWpjGS7LGkLg9q_ktU5OCZe-PEzYbMWj_sC1dYlmwH_vVcndSrf2vwudocaBn4f7pec5RxVYXKBaAShxYa5rK9rvbNjL6oh1PtoxJrhTLEaxBq7ngAft1MktgsWxcnkqnDgnV5A0G-iz-zKMe6TYnEBmFg5siOGBtsaVidAFKB5wQzlY188yCVIaHyhYYOCbKVMWAvnKYhTKBdUWTXjKg5BeoupiuYArhANumLAqNDLSLGRKaxNpTW1gORVhxrNrVHfvMvnaDceYFE9y87f4EZ30hu_JJHlN327RqVPBrvf1DlU3qy3co2PzvZmtVw-5bn8Ab1ieyQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=2017+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29&rft.atitle=DSAC+-+Differentiable+RANSAC+for+Camera+Localization&rft.au=Brachmann%2C+Eric&rft.au=Krull%2C+Alexander&rft.au=Nowozin%2C+Sebastian&rft.au=Shotton%2C+Jamie&rft.date=2017-07-01&rft.pub=IEEE&rft.issn=1063-6919&rft.eissn=1063-6919&rft.spage=2492&rft.epage=2500&rft_id=info:doi/10.1109%2FCVPR.2017.267&rft.externalDocID=8099750
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1063-6919&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1063-6919&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1063-6919&client=summon