Why an AI agent works one day and not the next: drift at the harness level

The agent was checked last week. It worked. This week it still reports progress, and it is confidently doing the wrong thing. That is not mood. It is agent drift: expected behaviour and actual behaviour have come apart, and the loop does not raise an error.

The instruction is still. The world under the agent is not.

What drift is

Drift is the gap that opens over time between what the operator asked for and what the agent actually does. The failure is quiet. From the outside it looks as if the model got worse. The model is usually the same. The soil under it has moved.

A research agent that keeps losing context between steps, a session that cannot be taken over, stale context that still drives the next action — these are the same class of failure. The session is alive. The job is not.

Why it happens

The world changes and the instruction does not. A site is redesigned. An API changes its payload. A vendor tool goes dark. The agent is still tuned for yesterday.

The environment fails silently. The browser, the mail gateway, the CRM connector dies. The agent does not halt. It idles, retries, or invents a path.

Working memory is lost. The process restarts, forgets where it was, and continues as if nothing happened. Lack of visibility into agent context is the operator’s first clue, and it usually arrives late.

Memory swells. The longer the run, the more residue it carries. Yesterday’s project leaks into today’s.

There is no independent check. The agent says done. Nothing moved. It is not lying in the human sense. It was never taught to verify its own work, so its self-report cannot be trusted.

How it looks in production

These are not prompt anecdotes. They are harness incidents from a live content and growth agent.

Sixteen minutes of idle work. The agent spun for sixteen minutes. It had not become less intelligent. Working memory was gone and the browser it acted through had dropped. The environment broke. The agent did not notice, and kept twitching.

The report is not the fact. A run announced “interrupted, unfinished.” The work was already done. Trust the report and the operator repeats a finished job. Close every scenario on the fact on the client side, not on what the process claimed.

Green inside, dead outside. The server was healthy. What the person saw was an infinite spinner. Internal signs said fine. The product was not usable. Check with the client’s eyes, not only with the host’s.

Projects bleed into each other. One agent, several parallel themes. Tasks piled up, memory bloated, and a fact from one book was pulled into another. From the outside: the agent got stupid. In the room: one heap, no index.

None of those cases was cured by a better prompt. The prompt is how the agent thinks. The harness is what stops it from sliding off without a sound.

The real boundary

The instinct is to retune the prompt. That plugs a hole. Next week it leaks somewhere else. A prompt is a fit to one moment. Drift is change across time.

Treat it as structure:

• Exit checks. The agent does not get to say done until an independent check confirms the result.
• Stop conditions. A tool is down, or retries are exhausted — halt. Do not improvise. That is what was written after the idle loop.
• Fact over self-report. Every scenario closes on a real effect the client can see.
• Structured memory. Split by context, index it, write and purge on a protocol. Memory without structure becomes a source of drift. Prompt rewriting does not fix a mixed heap.

The harness is not fire-and-forget. The world moves for it too. What it changes is the shape of failure: from invisible (the agent quietly goes the wrong way) to visible (the system reports the break). That is the difference between “sometimes fails without warning” and “fails rarely, and loudly.”

Reliability lives in the harness, not in the wording of the instruction.

The question to ask a contractor

If someone is building an AI system, one question is enough: what happens when the agent is wrong, or an external service drops? If the answer is about a well-tuned prompt, and nothing about checks, stop conditions, or result control, the system will break in the first month. The operator will find out late.

The useful question is not how to make the agent never err. That is not available. The useful question is how to keep the error from reaching the client unseen. That is harness engineering: the layer that turns “sometimes it works” into “when it fails, it fails in view.”