Variational Multiscale Nonparametric Regression: Algorithms and Implementation

Many modern statistically efficient methods come with tremendous computational challenges, often leading to large-scale optimisation problems. In this work, we examine such computational issues for recently developed estimation methods in nonparametric regression with a specific view on image denois...

Full description

Saved in:
Bibliographic Details
Published in:Algorithms Vol. 13; no. 11; p. 296
Main Authors: del Alamo, Miguel, Li, Housen, Munk, Axel, Werner, Frank
Format: Journal Article
Language:English
Published: Basel MDPI AG 01.11.2020
Subjects:
Algorithms
Constraints
Dictionaries
Estimators
Fines & penalties
image denoising
MIND estimator
Minimax technique
multiscale methods
Noise reduction
non-smooth large-scale optimisation
Nonparametric statistics
Optimization
Random variables
Regression analysis
Signal reconstruction
Smoothness
Statistical analysis
Statistical methods
variational estimation
ISSN:1999-4893, 1999-4893
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Many modern statistically efficient methods come with tremendous computational challenges, often leading to large-scale optimisation problems. In this work, we examine such computational issues for recently developed estimation methods in nonparametric regression with a specific view on image denoising. We consider in particular certain variational multiscale estimators which are statistically optimal in minimax sense, yet computationally intensive. Such an estimator is computed as the minimiser of a smoothness functional (e.g., TV norm) over the class of all estimators such that none of its coefficients with respect to a given multiscale dictionary is statistically significant. The so obtained multiscale Nemirowski-Dantzig estimator (MIND) can incorporate any convex smoothness functional and combine it with a proper dictionary including wavelets, curvelets and shearlets. The computation of MIND in general requires to solve a high-dimensional constrained convex optimisation problem with a specific structure of the constraints induced by the statistical multiscale testing criterion. To solve this explicitly, we discuss three different algorithmic approaches: the Chambolle-Pock, ADMM and semismooth Newton algorithms. Algorithmic details and an explicit implementation is presented and the solutions are then compared numerically in a simulation study and on various test images. We thereby recommend the Chambolle-Pock algorithm in most cases for its fast convergence. We stress that our analysis can also be transferred to signal recovery and other denoising problems to recover more general objects whenever it is possible to borrow statistical strength from data patches of similar object structure.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1999-4893
1999-4893
DOI:10.3390/a13110296