Interesting new benchmark: "Introducing Real-SWE"

<a href="https://news.ycombinator.com/item?id=49676820">Comments</a>

&#32; submitted by &#32; <a href="https://www.reddit.com/user/pmz"> /u/pmz </a> <br /> <span><a

<!-- SC_OFF --><div class="md"><p>Working on evaluating some AI-generated outbound (SDR-style emails along with follow-ups), and I’m running into a weird problem. Everyone talks

<!-- SC_OFF --><div class="md"><p>Figured this out by running 4 models: Claude Opus 4.6, GPT-5.4, Gemini 3.1 Pro, and Grok 4.20, on a benchmark of 1,417 binary forecasting

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s1brq3/wmb100k_open_source_benchmark_for_ai_memory/"> <img alt="WMB-100K – open source benchmark for AI

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1swfdbj/confirmed_swe_bench_is_now_a_benchmaxxed_benchmark/"> <img alt="Confirmed: SWE Bench is now a

<!-- SC_OFF --><div class="md"><p>ARC-AGI-3 is a deeply rigged benchmark and the marketing around it is insanely misleading - Human baseline is not “human,” it’s near-elite human

<!-- SC_OFF --><div class="md"><p>I know about DEEPSWE but it lacks many models :(</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/9r4n4y">