AI Evaluation is Becoming an Exciting Standalone Discipline
Introduction Throughout my academic career, I worked on various problems surrounding the robustness of deep learning models. Until a few years ago, these topics had their own niches within the academic but also industrial community. While some of this work got adopted in specific domains, e.g., consider corruption robustness benchmarks such as ImageNet-C in computer vision, much of it stayed separate from more mainstream developments such as foundation models. Today, in contrast, work on evaluation metrics and benchmarks […]