Hugging Face·· 2024-08-14AI 评分58
Hugging Face 复现 Infini-Attention 实验复盘：压缩注意力为何难以稳定扩展上下文
A failed experiment: Infini-Attention, and why we should keep trying?
AI 导读
Hugging Face 团队公布了复现 Google Infini-Attention 的实验结果，指出该机制随着内存压缩次数增加，长上下文检索性能会显著下降。
来源：Hugging Face · huggingface.co