Silent no-ops and fail-plausible: when the system reports success and does nothing

A catalog still serving last quarter’s prices, a risk job scoring a book on stale marks, a research agent that reports a finished brief while the source window never moved: the failure is not a 500. It is a successful run on dead content. This can escape detection because missing source and artifact checks can look like health: process up, logs clean, HTTP 200.

This is silent no-op — the loop accepted the work, wrote success, and changed nothing that matters. Its language-model cousin is fail-plausible: the system does not freeze, it narrates. Stale context can cause a related failure when a daemon keeps last week’s config, cache, or schema and never says so.

Green process, dead content

Ask how a team knows the automations still work. The answers cluster:

• Prometheus or Grafana: the process is running.
• Cron exited 0.
• Sentry is quiet.
• The endpoint returns 200 in 150 ms.

None of those facts is about the payload. A shop can keep serving frozen prices. A lead form can swallow requests after the mail gateway died. An assistant can answer from invented stock. The dashboard stays green because it is scoring that the program ran, not whether the content is true or current.

What one production study actually measured

Wu (arXiv:2606.14589) reports eight weeks in one personal-assistant runtime, with 22 documented incidents. The figures below are reported study findings, not our newly reproduced experiment or a rate for enterprise IT.

Finding	What it means for an operator
About 70% of silent failures were caught by a human looking at the output	Tests and health checks stayed green through most incidents. Reading the artifact is a control, not a luxury.
Retrospective governance: 0% ex-ante prevention, 87% later regression-block	In the paper’s retrospective audit of 15 incidents, later checks blocked known incident fixtures; this is not a universal prevention rate.
Silence lasted 13 hours to 60 days	The reported detection delays are from that runtime, not a general repair-time estimate.

The paper’s Class C is swallow-and-dilute. Class D is the LLM-specific escalation: the model turns an internal error into fluent narrative. That second class is why a freshness check on the file is not enough once a generator sits on the path.

Class C silence versus Class D fabrication

Class C — silence. Defensive code that must never crash:

try:
    data = fetch_external_prices()
except Exception:
    data = {}  # swallowed for "stability"

The token expired, the vendor moved, the array is []. Exit code 0. No alert.

Class D — fail-plausible. Give an agent empty or unauthorized source data and ask for a weekly brief. A model may fill gaps from training priors, stamp today’s date, and return a structured report. Freshness of the document looks fine. No number in it is tied to a live row.

Class C is treated with fail-loud plus a content-freshness gate. Class D needs an external physical trace: a hash of the source file, a confirmed diff in the database, a verbatim line the claim is allowed to rest on. Without that, you are grading prose.

Eight signatures of a silent no-op

1. The mtime trap

A watcher scores os.path.getmtime on the report file. Cron overwrites the file every 30 minutes, so the check is always green. Inside the file, the newest content date is two months old, copied from a stuck buffer. Check: file mtime alone does not establish source freshness. Inspect source observation time and payload dates against the task contract; old content can be valid when the source was checked and has not changed.

2. Schema drift without a version bump

A payment gateway or ERP drops or renames tax_amount and leaves the schema at v1.0. Consumers use data.get("tax_amount", 0.0). Nothing crashes. Tax lines go to zero. Invariant: strict contract validation (Pydantic, Zod) and no silent default on a field that moves money.

3. done (N chars)

The pipeline treats any non-empty string as success. The LLM gateway returns Error: Reached maximum number of turns — 45 characters — and that string is stored as the case summary. Invariant: separate payload from error text; require a minimum structure, not a minimum length.

4. Dead producer, evergreen consumer

The source service is decommissioned. The storefront catches Exception and returns []. The empty screen is 100% “available.” Invariant: an unexpected empty reply from a required producer fails visibly; an allowed no-result response must have explicit source-check evidence. Deleting a producer without a dependency audit is how this ships.

5. A watchman with no feed

Alert if drawdown exceeds 15%. The drawdown job stops running. Error count is zero, so the watchman never fires. Leadership reads “no risk.” Invariant: the watchman must also require a fresh key from the producer on a schedule, not only a threshold on a value that may not exist.

6. Config loaded outside the loop

config.json is read once before while True. An operator changes the live host or a safety threshold. The process runs for weeks on the old file. The running process has stale configuration: the loop still has a context; it is the wrong one. Invariant: reload on checksum change, or bounce the process on deploy. Do not assume the file on disk is the file in memory.

7. Calendar blindness

A risk or trading gate has a holiday calendar (FOMC, listed-exchange closures) and reports “safe day, no restriction” on a session it does not know about. Invariant: the system must declare the edge of its calendar and require an external calendar source, not a frozen table in the repo.

8. Fallback to a dead cache

On a transport blip the module substitutes last week’s snapshot instead of stopping. Customers order stock that is not there. Invariant: an expired TTL is an exception, not a substitute for live data.

A content-freshness gate

Before a scheduled job is allowed to call itself successful, three checks have to pass on the artifact, not on the process:

Invariant	Rule
Time	Source observation and payload dates meet the declared task-specific window
Volume	Required coverage is present; valid empty/no-change results have explicit rules
Change	A changed hash supports comparison; an unchanged hash needs source-check evidence, not an automatic failure

Apply the declared contract to distinguish failure, unknown and legitimate no-change. Missing required evidence must not be reported as a confirmed successful update. An unchanged source can be valid if it was actually checked.

Grade the artifact, not the path

It is a weak test to ask whether the agent called function X before function Y. A valid optimization breaks the chain; a determined model can mime the chain and still produce nothing. Score what left the sandbox: did the build run, did the order status change in the CRM, did the notification arrive on a real device.

A green dashboard is a claim. The checker that grades that claim cannot be the same loop that wrote the report.