SubQ - claims to be a different architecture - anyone tried?

On May 6th, a quiet question landed on LocalLLaMA: has anyone tried SubQ? The post scored 4.4 — enough to register, not enough to trend — and a second thread appeared the same day framing it as a potential "major breakthrough" architecture, though the scare quotes in the title did most of the editorial work. Both threads were thin on technical specifics, which is the usual texture of early community discovery: someone found something, isn't sure what to make of it, and throws it to the crowd. Then the Dev.to piece arrived and changed the shape of the conversation. That article scored 8.8 and gave the claim its clearest articulation yet: SubQ, short for Subquadratic, is positioning itself against the attention mechanism's core computational liability. Standard transformer attention scales quadratically with context length — double your tokens, quadruple your compute — which is the invisible tax that every long-context model is quietly paying. SubQ claims linear cost scaling and a context window of 12 million tokens. Those two numbers, placed side by side, are either a genuine architectural departure or the most optimistic benchmark framing since the beginning of the benchmark era. The architecture's name is doing some of the argumentative work here: subquadratic complexity isn't a new ambition — Mamba, RWKV, and various linear attention variants have all come for the same throne — but 12 million tokens at linear cost would represent a step change in what's practically deployable, not just theoretically possible.

If confirmed, here is what this means. The attention tax isn't a minor inefficiency — it's the reason long-context inference is expensive enough to shape product decisions, limit deployment contexts, and create an entire market segment around context compression. A genuinely linear architecture at 12 million tokens would hollow out that market overnight. Enterprises running retrieval-augmented pipelines built around chunking and summarisation would need to rethink the stack; the chunking exists specifically because full-context attention is prohibitive. Model serving costs would restructure, favouring whoever builds on the new substrate earliest. More strategically, it would reopen the hardware question: current GPU architectures are optimised around the memory access patterns of transformer attention, and a linear alternative could shift competitive advantage toward different silicon or different inference infrastructure entirely. The second-order effect is subtler but important — linear scaling changes what problems are even worth attempting. Tasks that require genuine long-range coherence across book-length or codebase-length contexts have been quietly set aside as economically impractical. They wouldn't be anymore.

Watch for independent benchmarks that test SubQ on tasks where quadratic attention cost actually bites — long-document QA, full-codebase reasoning — rather than the clean synthetic benchmarks that tend to flatter new architectures. Any response from established research groups, positive or skeptical, would clarify whether this is genuinely novel or a repackaging of ground already covered by the linear attention literature.

NewsHive monitors these sources continuously. All signal titles above link to the original reporting.

Intelligence by NewsHive. Need help navigating what this means for your business? Contact GeekyBee →