Craft
Why two people on the same model get different results.
The same prompt, two answers
Same model does not mean the same experiment
Give two people access to the same model and you can still get two very different verdicts. One sees a capable colleague. The other sees a fluent machine that misses the point. Both may be reporting their experience honestly. The mistake is assuming that the model name made their two tests identical.
A language model is probabilistic: it chooses among plausible next steps, so repeated runs can vary even when the visible prompt does not. A hosted service may also change routing, system instructions or the version behind a familiar label. Those variables sit outside the operator's control. But the prompt is only one part of the input. The surrounding context, available tools, success criteria and state of the session matter too, and those are often ours to improve.
So the useful question is not who is to blame? It is which conditions changed, and which of them can I control?
The model is one variable in the result
Two people can use the same engine while supplying different working conditions.
- Shared engine Model family, provider limits and release behaviour.
- Different task definition Outcome, constraints and examples change what the model attempts.
- Different evidence Files, decisions, freshness and source quality change what it can ground.
- Different harness Tools, permissions, tests and review determine what survives.
- Different result Quality varies even before individual judgement enters the comparison.
Compare the whole system before crediting or blaming the model.
The controllable part
The room changes what the model can use
Think of the model as one participant in a working environment. It can only reason over what reaches it, and the useful evidence may be competing with stale instructions, abandoned drafts and earlier guesses. A long context window is capacity, not a promise that every detail receives equal attention. As the window becomes noisy, relevant facts can become harder to use. That is one route into the dumbzone, where the prose can remain polished while the reasoning becomes less reliable.
The controllable response is not a magic prompt. It is a cleaner working environment: start a new session when the task changes, provide the smallest authoritative set of sources, say what a good answer must contain, and give the model a way to check its work. This is context engineering in practical form. It does not remove randomness, but it makes the conditions easier to inspect and improve.
A worked diagnosis
Change one condition at a time
Suppose you ask a model to review a supplier proposal and identify the commercial risks. On the first run, you open a clean session, attach the current proposal and pricing policy, and define risk as cost exposure, delivery dependency or an assumption without evidence. The answer is specific and useful.
A week later you type the same request at the end of a long thread. The session contains an older proposal, a discarded pricing assumption and several speculative notes. The answer is vague. That does not prove the model became worse, and it does not prove the operator failed. It tells you that the experiment changed.
Diagnose it by rebuilding the conditions. Open a clean session. Supply only the current documents. Reuse the same definition of risk. Run the task more than once and compare the answers against a short checklist. If quality returns, context was probably the main constraint. If the results still vary, inspect ambiguity in the request, differences in the service and the model's own run-to-run variation. The point is not to force certainty. It is to stop changing five variables and calling the outcome a verdict.
A practical rule
Separate the system from the conditions you set
When output disappoints you, check the layers in order. Confirm the model and settings you actually used. Reset the session. Reduce the context to current, relevant evidence. Define what success looks like. Then repeat the run before drawing a conclusion. This will not explain every difference, but it turns a complaint into a test.
A system that makes those conditions repeatable is what I mean by harness engineering. The harness is the context, tools, instructions, checks and boundaries around the model. It cannot guarantee identical answers from a probabilistic system. It can make good conditions the default and failures easier to investigate.
The useful conclusion
Variation is a reason to become a better student
The more I use these systems, the less willing I am to treat one impressive answer or one poor answer as proof of anything. A single run is an example. A repeated, controlled comparison is evidence. That is a more patient habit, but it is also a more professional one when other people's decisions depend on the result.
Some variance belongs to the model and the service behind it. Some belongs to the task and the environment we provide. Wisdom is knowing the boundary, improving the part we control, and leaving enough uncertainty in the answer to keep checking.
Same prompt is not the same experiment. Build better conditions, then judge the system by more than one run.