Kimi Linear: How Moonshot AI Built a Hybrid Attention Architecture That Beats Fu

arXiv:2603.22300v1 Announce Type: cross Abstract: Scaling Transformers to ultra-long contexts is bottlenecked by the $O(n^2 d)$ cost of self-attention. Existing methods reduce

<p>Last week, a Baseten inference engineer who goes by <a href="https://x.com/waterloo_intern/status/2081762065392541951" rel="noopener noreferrer">@waterloo_intern</a> published

<p>Every transformer you've ever used stacks layers with a dead-simple formula: take the input, add the layer's output, move on. <code>x + layer(x)</code>. Fixed weight of 1. No

arXiv:2604.05887v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have advanced unified reasoning over text, images, and videos, but their inference is

<a href="https://news.ycombinator.com/item?id=49082022">Comments</a>

<!-- SC_OFF --><div class="md"><p><em>Scaling language models to handle longer input sequences typically necessitates large key-value (KV) caches, resulting in substantial memory

<h2> The shift to hybrid reasoning architectures </h2> <p>For the past two years, engineering teams building production generative AI systems operated under a rigid architectural

arXiv:2608.23834v1 Announce Type: new Abstract: The key-value (KV) cache is a primary capacity and bandwidth bottleneck in long-context LLM serving. We present Minima-KV, a