Prediction-Powered Active Testing

arXiv:2607.08347v1 Announce Type: new
Abstract: Active testing provides a label–efficient approach to risk estimation by adaptively selecting which test points should be labelled. However, existing estimators fail to exploit the informative predictions of powerful black–box models, even though such predictions are increasingly available in settings where labels remain expensive. To address this, we propose textbf{Prediction–Powered Active Testing (PPAT)}, a novel label–efficient risk estimation framework that combines the unbiased LURE estimator citep{farquhar2021statistical} with a prediction–powered control variate. Rather than using proxy predictions as biased pseudo–labels, PPAT uses them to residualise the loss, preserving unbiasedness while reducing variance. Beyond the estimator itself, PPAT also changes which points should be acquired: we derive oracle and practical surrogate–based acquisition rules tailored to reducing the variance of our estimator. Moreover, we establish asymptotic normality for PPAT, yielding asymptotically valid confidence intervals and thus a principled estimate of the uncertainty around our estimates. Across tabular regression and image–classification tasks, PPAT outperforms existing methods in risk estimation, while its confidence intervals attain the target coverage with substantially fewer labels and smaller widths.