Scaling Laws for Masked-Reconstruction Transformers on Single-Cell Transcriptomics
arXiv:2602.15253v1 Announce Type: new Abstract: Neural scaling laws — power-law relationships between loss, model size, and data — have been extensively documented for language and vision transformers, yet their existence in single-cell genomics remains largely unexplored. We present the first systematic study of scaling behaviour for masked-reconstruction transformers trained on single-cell RNA sequencing (scRNA-seq) data. Using expression profiles from the CELLxGENE Census, we construct two experimental regimes: a data-rich regime (512 highly variable genes, 200,000 cells) and a […]