Prometheus Mind: Retrofitting Memory to Frozen Language Models

arXiv:2601.15324v1 Announce Type: new
Abstract: Adding memory to pretrained language models typically requires architectural changes or weight modification. We present Prometheus Mind, which retrofits memory to a frozen Qwen3-4B using 11 modular adapters (530MB, 7% overhead) — fully reversible by removing the adapters. Building this system required solving four problems: (1) Extraction — we develop Contrastive Direction Discovery (CDD), which finds semantic directions via minimal pairs without labeled data. (2) Training — end-to-end optimization collapses; stage-wise training of each adapter on simple proxy tasks succeeds. (3) Injection — learned encoders fail to generalize; we find that lm_head.weight rows already provide the mapping we need, requiring no training. (4) Hidden state collapse — transformers make “wife” and “brother” 0.98+ similar; we train projections to recover distinction (0.98 $rightarrow$ 0.09). On PrometheusExtract-132 (132 cases), the system achieves 94.4% retrieval on clean inputs (n=54, 95% CI: [84.9%, 98.1%]), degrading to 19.4% on informal inputs with ellipsis, filler words, or implicit subjects (n=36). The primary bottleneck is relation classification (47.3% accuracy), responsible for most extraction errors.