Relaxed Triangle Inequality for Kullback-Leibler Divergence Between Multivariate Gaussian Distributions
arXiv:2602.02577v1 Announce Type: new
Abstract: The Kullback-Leibler (KL) divergence is not a proper distance metric and does not satisfy the triangle inequality, posing theoretical challenges in certain practical applications. Existing work has demonstrated that KL divergence between multivariate Gaussian distributions follows a relaxed triangle inequality. Given any three multivariate Gaussian distributions $mathcal{N}_1, mathcal{N}_2$, and $mathcal{N}_3$, if $KL(mathcal{N}_1, mathcal{N}_2)leq epsilon_1$ and $KL(mathcal{N}_2, mathcal{N}_3)leq epsilon_2$, then $KL(mathcal{N}_1, mathcal{N}_3)< 3epsilon_1+3epsilon_2+2sqrt{epsilon_1epsilon_2}+o(epsilon_1)+o(epsilon_2)$. However, the supremum of $KL(mathcal{N}_1, mathcal{N}_3)$ is still unknown. In this paper, we investigate the relaxed triangle inequality for the KL divergence between multivariate Gaussian distributions and give the supremum of $KL(mathcal{N}_1, mathcal{N}_3)$ as well as the conditions when the supremum can be attained. When $epsilon_1$ and $epsilon_2$ are small, the supremum is $epsilon_1+epsilon_2+sqrt{epsilon_1epsilon_2}+o(epsilon_1)+o(epsilon_2)$. Finally, we demonstrate several applications of our results in out-of-distribution detection with flow-based generative models and safe reinforcement learning.
