TRACER: Learn-to-Defer for LLM Classification with Formal Teacher-Agreement Guarantees
|
I’m releasing TRACER (Trace-Based Adaptive Cost-Efficient Routing), a library for learning cost-efficient routing policies from LLM traces. The setup: you have an LLM handling classification tasks. You want to replace a fraction of calls with a cheap local surrogate, with a formal guarantee that the surrogate agrees with the LLM at least X% of the time on handled traffic. Technical core:
Results on Banking77 (77-class intent, BGE-M3 embeddings):
Paper in progress. Feedback welcome. submitted by /u/Adr-740 |
Like
0
Liked
Liked