The Model Did Not Change: The Test Was Comparing Strings, Not Behavior

<p>A patch touches zero test files. No assertion changes, no new skips, no fixture edits. The suite goes green on the first run.</p> <p>Then someone opens the job duration graph

<p>At 2:13 a.m. last Tuesday, my GitLab job went red. The stack trace ended with <code>404: /v1/chat/completions not found</code>.</p> <p>I checked my code. Nothing had

<p>Add retries only after the request knows what it already did. Free model endpoints and free servers fail in the worst place: after the work may have run but before you receive

<p>Silent LLM regressions survive snapshot tests because the payload still parses and the last message still looks fluent. A harness that versions golden behaviors and scores them