Distributional Shrinkage I: Universal Denoiser Beyond Tweedie’s Formula

arXiv:2511.09500v4 Announce Type: replace
Abstract: We study the problem of denoising when only the noise level is known, not the noise distribution. Independent noise $Z$ corrupts a signal $X$, yielding the observation $Y = X + sigma Z$ with known $sigma in (0,1)$. We propose emph{universal} denoisers, agnostic to both signal and noise distributions, that recover the signal distribution $P_X$ from $P_Y$. When the focus is on distributional recovery of $P_X$ rather than on individual realizations of $X$, our denoisers achieve order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie’s formula, which achieves $O(sigma^2)$ accuracy. They shrink $P_Y$ toward $P_X$ with $O(sigma^4)$ and $O(sigma^6)$ accuracy in matching generalized moments and densities. Drawing on optimal transport theory, our denoisers approximate the Monge–Amp`ere equation with higher-order accuracy and can be implemented efficiently via score matching.
Let $q$ denote the density of $P_Y$. For distributional denoising, we propose replacing the Bayes-optimal denoiser, $$mathbf{T}^*(y) = y + sigma^2 nabla log q(y),$$ with denoisers exhibiting less-aggressive distributional shrinkage, $$mathbf{T}_1(y) = y + frac{sigma^2}{2} nabla log q(y),$$ $$mathbf{T}_2(y) = y + frac{sigma^2}{2} nabla log q(y) – frac{sigma^4}{8} nabla !left( frac{1}{2} | nabla log q(y) |^2 + nabla cdot nabla log q(y) right)!.$$