It’s the pipeline, not the model

Most inconsistency comes from retrieval returning different context, prompts drifting between versions, and tools failing silently.

Pin everything

Version your prompts, your retrieval settings and your tool schemas together, so a change in one is visible in your evals.

Measure variance directly

Run the same input ten times. If the answers differ in ways your users would notice, you have found your next fix.