[D] Matryoshka Representation Learning

Hey everyone,

Matryoshka Representation Learning (MRL) has gained a lot of traction for its ability to maintain strong downstream performance even under aggressive embedding compression. That said, I’m curious about its limitations.

While I’ve come across some recent work highlighting degraded performance in certain retrieval-based tasks, I’m wondering if there are other settings where MRL struggles.

Would love to hear about any papers, experiments, or firsthand observations that explore where MRL falls short.

Link to MRL paper – https://arxiv.org/abs/2205.13147

Thanks!