SMat-Attention: Structured Long-Context Sequence Modeling

arXiv:2603.22300v1 Announce Type: cross Abstract: Scaling Transformers to ultra-long contexts is bottlenecked by the $O(n^2 d)$ cost of self-attention. Existing methods reduce

arXiv:2603.18031v1 Announce Type: cross Abstract: Balancing fine-grained local modeling with long-range dependency capture under computational constraints remains a central

arXiv:2609.38385v1 Announce Type: new Abstract: Supervised fine-tuning of discrete diffusion language models masks some response tokens and trains the model to recover their

<!-- SC_OFF --><div class="md"><p><em>Scaling language models to handle longer input sequences typically necessitates large key-value (KV) caches, resulting in substantial memory

arXiv:2604.09852v1 Announce Type: new Abstract: Reasoning models think in long, unstructured streams with no mechanism for compressing or organizing their own intermediate state.