Distributional Shrinkage I: Universal Denoiser Beyond Tweedie’s Formula
arXiv:2511.09500v3 Announce Type: replace
Abstract: We revisit the problem of denoising from noisy measurements where only the noise level is known, not the noise distribution. In multi-dimensions, independent noise $Z$ corrupts the signal $X$, resulting in the noisy measurement $Y = X + sigma Z$, where $sigma in (0, 1)$ is a known noise level. Our goal is to recover the underlying signal distribution $P_X$ from denoising $P_Y$. We propose and analyze universal denoisers that are agnostic to a wide range of signal and noise distributions. Our distributional denoisers offer order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie’s formula, if the focus is on the entire distribution $P_X$ rather than on individual realizations of $X$. Our denoisers shrink $P_Y$ toward $P_X$ optimally, achieving $O(sigma^4)$ and $O(sigma^6)$ accuracy in matching generalized moments and density functions. Inspired by optimal transport theory, the proposed denoisers are optimal in approximating the Monge-Amp`ere equation with higher-order accuracy, and can be implemented efficiently via score matching.
Let $q$ represent the density of $P_Y$; for optimal distributional denoising, we recommend replacing the Bayes-optimal denoiser, [ mathbf{T}^*(y) = y + sigma^2 nabla log q(y), ] with denoisers exhibiting less aggressive distributional shrinkage, [ mathbf{T}_1(y) = y + frac{sigma^2}{2} nabla log q(y), ] [ mathbf{T}_2(y) = y + frac{sigma^2}{2} nabla log q(y) – frac{sigma^4}{8} nabla left( frac{1}{2} | nabla log q(y) |^2 + nabla cdot nabla log q(y) right) . ]
