ICML 2026 Review Policy Comparison

The debate opened on March 25th with a Reddit thread comparing two review policy approaches — Policy A and Policy B — and asking a simple, pointed question: does the policy you use change the scores papers receive, or just the confidence reviewers express? That framing matters. If policy mechanics can move outcomes independently of paper quality, then the review process has a fairness problem that no amount of careful writing can fix. The thread found an audience, which is itself worth noting — ML researchers are not usually drawn to procedural debates unless they suspect the procedure is doing something to them.

Six days later, on March 31st, a follow-up arrived with actual numbers: roughly 100 survey responses suggesting Policy B tends to produce higher scores, while Policy A correlates with higher reviewer confidence. That is a genuinely interesting asymmetry, if it holds. Higher scores without higher confidence could mean reviewers are being nudged toward generosity by the structure of the form itself. April brought the anxiety into sharper relief. A comparative analysis of ICLR 2025 versus 2026 scores drew a surprised reaction from the community — the "WOW" in the thread title doing real work — while a separate post described polarising ICML reviews leaving authors genuinely confused about where they stood. Then, on April 15th, someone looked at an ICLR 2025 oral paper and reported being shocked it had reached that tier at all. By April 30th, the thread titles had stopped being analytical and started being alarmed: ICML was apparently rejecting papers with unanimous positive ratings. On May 1st, the final decisions arrived and someone used the word "rant" in their thread title. On May 6th, the conversation drifted toward NeurIPS 2026, which is piloting an AC-assisted review process — and someone was already asking how much trust to place in it.

If confirmed, here is what this means. Academic ML conferences are the gatekeeping infrastructure for the entire field — they determine what gets published, what gets cited, what gets funded, and whose career advances. If review policy mechanics are systematically inflating or suppressing scores independent of paper quality, then the outcomes being produced right now are structurally unreliable, and everyone building on them is standing on uncertain ground. The rejection of unanimously positively-rated papers is the detail that cuts deepest: it suggests area chairs or program chairs are overriding reviewer consensus, either because they distrust the scores or because they are managing acceptance rates. Neither explanation is reassuring. The NeurIPS AC-Pilot signal adds a second-order dimension — if the community is already skeptical of automated assistance in a process they already distrust, adoption will be slow and contested regardless of how well the technology actually performs.

Watch for any official statement from ICML 2026 program chairs addressing the rejection rate or the policy comparison — and watch for whether the NeurIPS AC-Pilot generates structured evaluation data rather than more Reddit threads.

NewsHive monitors these sources continuously. All signal titles above link to the original reporting.

Intelligence by NewsHive. Need help navigating what this means for your business? Contact GeekyBee →