Attention Degradation, Function Token Anchoring, and the Limits of Attention-Bas

arXiv:2603.22300v1 Announce Type: cross Abstract: Scaling Transformers to ultra-long contexts is bottlenecked by the $O(n^2 d)$ cost of self-attention. Existing methods reduce

arXiv:2605.12922v1 Announce Type: new Abstract: Large language models can follow complex instructions in a single turn, yet over long multi-turn interactions they often lose the

arXiv:2606.27793v1 Announce Type: cross Abstract: Long-context language models suffer from position bias, where information in middle positions is underutilized. Attention Sorting

<!-- SC_OFF --><div class="md"><p><em>Scaling language models to handle longer input sequences typically necessitates large key-value (KV) caches, resulting in substantial memory

arXiv:2610.21483v1 Announce Type: new Abstract: Large language models can produce fluent answers that the supplied context does not support. A compact frozen proxy model can

arXiv:2603.22608v1 Announce Type: new Abstract: Users often rely on Large Language Models (LLMs) for processing multiple documents or performing analysis over a number of

arXiv:2605.01214v1 Announce Type: new Abstract: This position paper argues that agentic AI systems should be designed and evaluated as \emph{marginal token allocation economies}

<blockquote> <p><em>Originally published at <a href="https://adiyogiarts.com/blog/identifying-early-warning-signs-of-attention-mecha" rel="noopener