NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images

Neural Radiance Fields (NeRF) is a technique for high quality novel view synthesis from a collection of posed input images. Like most view synthesis methods, NeRF uses tonemapped low dynamic range (LDR) as input; these images have been processed by a lossy camera pipeline that smooths detail, clips...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 16169 - 16178
Main Authors: Mildenhall, Ben, Hedman, Peter, Martin-Brualla, Ricardo, Srinivasan, Pratul P., Barron, Jonathan T.
Format: Conference Proceeding
Language:English
Published: IEEE 01.01.2022
Subjects:
3D from multi-view and sensors; Computational photography; Low-level vision; Vision + graphics
Cameras
Dynamic range
Pattern recognition
Photography
Pipelines
Rendering (computer graphics)
Three-dimensional displays
ISSN:1063-6919
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Neural Radiance Fields (NeRF) is a technique for high quality novel view synthesis from a collection of posed input images. Like most view synthesis methods, NeRF uses tonemapped low dynamic range (LDR) as input; these images have been processed by a lossy camera pipeline that smooths detail, clips highlights, and distorts the simple noise distribution of raw sensor data. We modify NeRF to instead train directly on linear raw images, preserving the scene's full dynamic range. By rendering raw output images from the resulting NeRF, we can perform novel high dynamic range (HDR) view synthesis tasks. In addition to changing the camera viewpoint, we can manipulate focus, exposure, and tonemapping after the fact. Although a single raw image appears significantly more noisy than a postprocessed one, we show that NeRF is highly robust to the zeromean distribution of raw noise. When optimized over many noisy raw inputs (25-200), NeRF produces a scene representation so accurate that its rendered novel views outperform dedicated single and multi-image deep raw denoisers run on the same wide baseline input images. As a result, our method, which we call RawNeRF, can reconstruct scenes from extremely noisy images captured in near-darkness.
AbstractList Neural Radiance Fields (NeRF) is a technique for high quality novel view synthesis from a collection of posed input images. Like most view synthesis methods, NeRF uses tonemapped low dynamic range (LDR) as input; these images have been processed by a lossy camera pipeline that smooths detail, clips highlights, and distorts the simple noise distribution of raw sensor data. We modify NeRF to instead train directly on linear raw images, preserving the scene's full dynamic range. By rendering raw output images from the resulting NeRF, we can perform novel high dynamic range (HDR) view synthesis tasks. In addition to changing the camera viewpoint, we can manipulate focus, exposure, and tonemapping after the fact. Although a single raw image appears significantly more noisy than a postprocessed one, we show that NeRF is highly robust to the zeromean distribution of raw noise. When optimized over many noisy raw inputs (25-200), NeRF produces a scene representation so accurate that its rendered novel views outperform dedicated single and multi-image deep raw denoisers run on the same wide baseline input images. As a result, our method, which we call RawNeRF, can reconstruct scenes from extremely noisy images captured in near-darkness.
Author Martin-Brualla, Ricardo
Mildenhall, Ben
Hedman, Peter
Srinivasan, Pratul P.
Barron, Jonathan T.
Author_xml – sequence: 1
  givenname: Ben
  surname: Mildenhall
  fullname: Mildenhall, Ben
  organization: Google Research
– sequence: 2
  givenname: Peter
  surname: Hedman
  fullname: Hedman, Peter
  organization: Google Research
– sequence: 3
  givenname: Ricardo
  surname: Martin-Brualla
  fullname: Martin-Brualla, Ricardo
  organization: Google Research
– sequence: 4
  givenname: Pratul P.
  surname: Srinivasan
  fullname: Srinivasan, Pratul P.
  organization: Google Research
– sequence: 5
  givenname: Jonathan T.
  surname: Barron
  fullname: Barron, Jonathan T.
  organization: Google Research
BookMark eNotjs1OwkAURkejiYA8gS7mBVrvnelM77gz_AgJQVOVLRnaC4zaYjokpG8via6-xTk5-friqjk0LMQ9QooI7mG0ei2MskSpAqVSQJPjheijtSazLrP6UvQQrE6sQ3cjhjF-AoBWiNZRT8yXXExlaORxz3Ls269HOQu7vRx3ja9DKQvf7FiuAp_kW9ecpRii3LaHWi4PIXZnfpLz2u843orrrf-OPPzfgfiYTt5Hs2Tx8jwfPS2SoKw7Joq8ovNfVVl0FeVuo_RmQ-SAHGPlUW-tU0xVBghgqhKwNIaArQEqTaUH4u6vG5h5_dOG2rfd2lFOmcn1LwnQTGM
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/CVPR52688.2022.01571
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE/IET Electronic Library
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Applied Sciences
EISBN 1665469463
9781665469463
EISSN 1063-6919
EndPage 16178
ExternalDocumentID 9878457
Genre orig-research
GroupedDBID 6IE
6IH
6IL
6IN
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IJVOP
OCL
RIE
RIL
RIO
ID FETCH-LOGICAL-i269t-28a282682d619d879b23bb889089e1da13f692e8d401005dc01c5580e6508c5d3
IEDL.DBID RIE
ISICitedReferencesCount 221
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000870783001094&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:15:09 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i269t-28a282682d619d879b23bb889089e1da13f692e8d401005dc01c5580e6508c5d3
PageCount 10
ParticipantIDs ieee_primary_9878457
PublicationCentury 2000
PublicationDate 2022-01-01
PublicationDateYYYYMMDD 2022-01-01
PublicationDate_xml – month: 01
  year: 2022
  text: 2022-01-01
  day: 01
PublicationDecade 2020
PublicationTitle Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online)
PublicationTitleAbbrev CVPR
PublicationYear 2022
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0003211698
Score 2.647411
Snippet Neural Radiance Fields (NeRF) is a technique for high quality novel view synthesis from a collection of posed input images. Like most view synthesis methods,...
SourceID ieee
SourceType Publisher
StartPage 16169
SubjectTerms 3D from multi-view and sensors; Computational photography; Low-level vision; Vision + graphics
Cameras
Dynamic range
Pattern recognition
Photography
Pipelines
Rendering (computer graphics)
Three-dimensional displays
Title NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images
URI https://ieeexplore.ieee.org/document/9878457
WOSCitedRecordID wos000870783001094&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Lb8IwDLYA7bAT22DaWznsuEKTPpLsCkNDmirENsQNtUkqVdMKojDEv1_cVnDZZbeoqVLJlWN_tj8b4FEyLZKUeo5OlUKAQh1BLVhJMWvlUZ_yuOyu_8ajSMznctKApwMXxhhTFp-ZHi7LXL5eqi2GyvoWHws_4E1och5WXK1DPMWzSCaUombHUVf2B7PJFJuZYAEXYz1r90qe_HGGSmlCRu3_ffwMukcuHpkcrMw5NEx-Ae3aeSS1ahYdGEdmOiJZTqxHR4bx-uuZYA0HGVYj58kUWQRklpkded_n9qUiKwiSS0i0zIq93d-R8be9XYoufI5ePgavTj0nwclYKDcOE7EFTqFg2qIhLbhMmJckQmBKz1AdUy8NJTNCWyxllU4rl6ogEK5B70wF2ruEVr7MzRWQVKaucqWQ1Cg_ta6O4n7ME-xQo-xB8ho6KJnFqmqFsaiFcvP341s4RdFXEYs7aG3WW3MPJ-pnkxXrh_L__QIEDpj6
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEJ4gmugJFYxve_Dowrb7ar2CBCJuCCLhRnbbbrIxLoYFCf_ezrKBixdvTdu0SZt25puZbwbgUTDF44Q6lkqkRIBCLU4NWEnQa-VQlwZRkV1_EIQhn07FsAJPOy6M1roIPtNNbBa-fDWXKzSVtQw-5q4XHMAhVs4q2Vo7i4pjsIwveMmPo7ZotSfDEaYzwRAuxppG8hVM-X0VlUKIdGv_2_4UGns2Hhnu5MwZVHR2DrVSfSTl48zr0A_1qEvSjBidjnSixeczwSgO0tkWnScj5BGQSarX5H2TmUl5mhOkl5BwnuYbM74m_S_zv-QN-Oi-jNs9q6yUYKXMF0uL8chAJ58zZfCQ4oGImRPHnKNTT1MVUSfxBdNcGTRlnp2SNpWex22N-pn0lHMB1Wye6UsgiUhsaQsuqJZuYpQdGbhREGOOGmkWEldQx5OZfW-TYczKQ7n-u_sBjnvjt8Fs0A9fb-AEr2Frv7iF6nKx0ndwJH-Wab64L-7yFzATnEM
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=NeRF+in+the+Dark%3A+High+Dynamic+Range+View+Synthesis+from+Noisy+Raw+Images&rft.au=Mildenhall%2C+Ben&rft.au=Hedman%2C+Peter&rft.au=Martin-Brualla%2C+Ricardo&rft.au=Srinivasan%2C+Pratul+P.&rft.date=2022-01-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=16169&rft.epage=16178&rft_id=info:doi/10.1109%2FCVPR52688.2022.01571&rft.externalDocID=9878457