A Visual Guide to Attention Variants in Modern LLMs
From MHA and GQA to MLA, sparse attention, and hybrid architectures
Like
0
Liked
Liked
From MHA and GQA to MLA, sparse attention, and hybrid architectures