How to Build a Benchmark That Actually Measures Taste

arXiv:2610.02599v1 Announce Type: new Abstract: Sustainable protein discovery lacks the fast computational proxies, analogous to molecular docking or density functional theory,

arXiv:2604.22436v1 Announce Type: new Abstract: The rapid growth of AI agent ecosystems is transforming how complex tasks are delegated and executed, creating a new challenge of

arXiv:2603.18019v1 Announce Type: cross Abstract: Do language model benchmarks actually measure what practitioners intend them to ? High-level metadata is too coarse to convey the

<!-- SC_OFF --><div class="md"><p>Projects are <a href="https://github.com/snap-research/locomo/issues/31">still submitting new scores on LoCoMo as of March 2026.</a> but the

arXiv:2609.02067v1 Announce Type: new Abstract: Scientific benchmarks are commonly built by domain experts who write tasks and cross-check one another's work, or who adapt

arXiv:2603.22305v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly transitioning from static Natural Language Processing (NLP) tasks including sentiment

arXiv:2604.25937v1 Announce Type: cross Abstract: Recent advancements in Text-to-Song generation have enabled realistic musical content production, yet existing evaluation

<blockquote> <p>A controlled study finds that data distribution, not just volume, dictates LLM capability. Benchmark-aligned training inflates scores but creates narrow, brittle