Probability is Earned: Information Capacity and the Epistemic Geometry of Inference
Probability is often treated as the default representation of uncertainty in statistical inference and machine learning. This paper asks a more fundamental question: under what conditions is a probability distribution a valid representation of uncertainty, and what is the information cost of assuming one when those conditions are not met? We show that inference is governed by two joint constraints: maximizing information capacity by preserving the geometric degrees of freedom through which contrast can register, and minimizing false information by asserting nothing the evidence has not forced. These constraints, expressed through Jaynes’s principle of maximum entropy and Popper’s criterion of falsification, determine the structure of inference without remainder. Bayesian inference emerges not as a competing framework, but as the limiting geometry obtained when epistemic width has contracted sufficiently to justify probabilistic closure. In this sense, probability is not assumed—it is earned. We trace the origin of these ideas through two decades of operational experience in spacecraft navigation, space situational awareness, and orbit determination, where standard probabilistic filters performed well in nominal regimes but failed systematically when uncertainty was driven by genuine ignorance rather than statistical variability. Across problems including debris tracking, attitude estimation, and multi-target inference, the consistent failure mode was premature probabilistic commitment in regimes where observation geometry could not support distinguishability. The central result is that information exists only in the presence of contrast, and that structure destroyed without evidence justification is information permanently lost. We formalize this principle through an epistemic geometry of inference and show that probabilistic representations are valid only when distinguishability, parameterization, and likelihood structure are all earned by the data. When these conditions fail, probabilistic closure incurs a measurable and avoidable information capacity cost.
