Shift schema drift left: policy-aware compile-time contracts for typed JVM and Spark pipelines
arXiv:2604.16986v1 Announce Type: new Abstract: Schema drift in data pipelines is often caught only when a job touches real data. Typed-Dataset layers close part of this gap but require wholesale adoption; table-level enforcement systems close another part but operate at write time against a stored schema. We present a small Scala 3 framework that occupies the seam: it proves producer-to-contract structural compatibility under explicit policies at compile time, derives Spark schemas from the same contract types, and re-checks […]