Silent no-ops and fail-plausible: when the system reports success and does nothing

A catalog still serving last quarter’s prices, a risk job scoring a book on stale marks, a research agent that reports a finished brief while the source window never moved: the failure is not a 500. It is a successful run on dead content. Operators lose weeks to this because lack of visibility into agent context looks like health: process up, logs clean, HTTP 200.

This is silent no-op — the loop accepted the work, wrote success, and changed nothing that matters. Its language-model cousin is fail-plausible: the system does not freeze, it narrates. Agent drift due to stale context is the same family when a daemon keeps last week’s config, cache, or schema and never says so.

Green process, dead content

Ask how a team knows the automations still work. The answers cluster:

• Prometheus or Grafana: the process is running.
• Cron exited 0.
• Sentry is quiet.
• The endpoint returns 200 in 150 ms.

None of those facts is about the payload. A shop can keep serving frozen prices. A lead form can swallow requests after the mail gateway died. An assistant can answer from invented stock. The dashboard stays green because it is scoring that the program ran, not whether the content is true or current.

What one production study actually measured

Wu et al. (arXiv:2606.14589) followed a live LLM-agent runtime for eight weeks: about 40 scheduled jobs, thousands of unit tests, hundreds of governance checks. The useful numbers are about discovery, not about “all of enterprise IT”:

Finding	What it means for an operator
About 70% of silent failures were caught by a human looking at the output	Tests and health checks stayed green through most incidents. Reading the artifact is a control, not a luxury.
Retrospective governance: 0% ex-ante prevention, 87% later regression-block	You cannot write the next novel failure in advance. You can encode the last one so it cannot hide twice.
Silence lasted 13 hours to 60 days	A crash is repaired in minutes. A quiet freeze can distort decisions for a quarter.

The paper’s Class C is swallow-and-dilute. Class D is the LLM-specific escalation: the model turns an internal error into fluent narrative. That second class is why a freshness check on the file is not enough once a generator sits on the path.

Class C silence versus Class D fabrication

Class C — silence. Defensive code that must never crash:

try:
    data = fetch_external_prices()
except Exception:
    data = {}  # swallowed for "stability"

The token expired, the vendor moved, the array is []. Exit code 0. No alert.

Class D — fail-plausible. Give an agent empty or unauthorized source data and ask for a weekly brief. The model is trained to continue. It fills gaps from training priors, stamps today’s date, and returns a structured report. Freshness of the document looks fine. No number in it is tied to a live row.

Class C is treated with fail-loud plus a content-freshness gate. Class D needs an external physical trace: a hash of the source file, a confirmed diff in the database, a verbatim line the claim is allowed to rest on. Without that, you are grading prose.

Eight signatures of a silent no-op

1. The mtime trap

A watcher scores os.path.getmtime on the report file. Cron overwrites the file every 30 minutes, so the check is always green. Inside the file, the newest content date is two months old, copied from a stuck buffer. Invariant: mtime is not evidence. Parse max(content_date).

2. Schema drift without a version bump

A payment gateway or ERP drops or renames tax_amount and leaves the schema at v1.0. Consumers use data.get("tax_amount", 0.0). Nothing crashes. Tax lines go to zero. Invariant: strict contract validation (Pydantic, Zod) and no silent default on a field that moves money.

3. done (N chars)

The pipeline treats any non-empty string as success. The LLM gateway returns Error: Reached maximum number of turns — 45 characters — and that string is stored as the case summary. Invariant: separate payload from error text; require a minimum structure, not a minimum length.

4. Dead producer, evergreen consumer

The source service is decommissioned. The storefront catches Exception and returns []. The empty screen is 100% “available.” Invariant: an empty reply from a required producer is a loud failure. Deleting a producer without a dependency audit is how this ships.

5. A watchman with no feed

Alert if drawdown exceeds 15%. The drawdown job stops running. Error count is zero, so the watchman never fires. Leadership reads “no risk.” Invariant: the watchman must also require a fresh key from the producer on a schedule, not only a threshold on a value that may not exist.

6. Config loaded outside the loop

config.json is read once before while True. An operator changes the live host or a safety threshold. The process runs for weeks on the old file. That is agent drift due to stale context in daemon form: the loop still has a context; it is the wrong one. Invariant: reload on checksum change, or bounce the process on deploy. Do not assume the file on disk is the file in memory.

7. Calendar blindness

A risk or trading gate has a holiday calendar (FOMC, listed-exchange closures) and reports “safe day, no restriction” on a session it does not know about. Invariant: the system must declare the edge of its calendar and require an external calendar source, not a frozen table in the repo.

8. Fallback to a dead cache

On a transport blip the module substitutes last week’s snapshot instead of stopping. Customers order stock that is not there. Invariant: an expired TTL is an exception, not a substitute for live data.

A content-freshness gate

Before a scheduled job is allowed to call itself successful, three checks have to pass on the artifact, not on the process:

Invariant	Rule
Time	max(content_date) is today (or the last business session), not the file’s birth time
Volume	Row count is above a floor and not frozen (delta != 0 on a live feed)
Change	hash(today) != hash(yesterday) on a dataset that is supposed to move

If any check fails, the run is a critical alert to the on-call, not a green exit 0.

Grade the artifact, not the path

It is a weak test to ask whether the agent called function X before function Y. A valid optimization breaks the chain; a determined model can mime the chain and still produce nothing. Score what left the sandbox: did the build run, did the order status change in the CRM, did the notification arrive on a real device.

A green dashboard is a claim. The checker that grades that claim cannot be the same loop that wrote the report.