Agent Reliability LabDiscuss your needs
Materials

Written as operators, not as a feed.

Short notes from the same practice that audits production agents. Filter by cluster without scripts: each cluster is its own page.

AllVerificationReliabilityLocal modelsAutomationWeb
Verification

2026-10-20 · 7 min · Verification

Backtest Data Bugs: An Audit Sheet for Agent-Built AnalysisAudit backtest inputs for future data, missing values, expiry and time units. Use a runnable fictional fixture and a copyable defect sheet.Read →
Verification

2026-10-19 · 7 min · Verification

An API Responded: Does It Cover the Data You Need?Check API data completeness with a coverage matrix, pagination evidence and a worked Search Console example. Keep unknown data out of false totals.Read →
Automation

2026-10-15 · 6 min · Automation

n8n Error Workflow Not Triggering? A Failure-Handling ChecklistCheck n8n Error Trigger behaviour, linked error workflows and saved execution evidence. Use a failure checklist before retrying a workflow.Read →
Verification

2026-10-15 · 6 min · Verification

RAG Acceptance Checklist: A Test Sheet for Document AnswersCheck a document assistant with separate retrieval, answer and refusal tests. Copy an acceptance sheet with evidence and a worked example.Read →
Verification

2026-10-09 · 5 min · Verification

Scheduled AI Task Ran but No Result? What to Check FirstFind the run, locate its output and check delivery before retrying a scheduled AI task. A practical checklist and one recorded delivery timeout.Read →
Local models

2026-10-07 · 6 min · Local models

A Binary Answer Is Not a Simple Decision: Testing a Specialist AI ModelAn eight-case comparison of a specialist decision model and general models: expert agreement, missing answers, and what the test does not establish.Read →
Reliability

2026-10-01 · 4 min · Reliability

Telegram AI Bot Security: Four Controls Beyond the PromptProtect Telegram AI bot actions with backend policy, scoped permissions, minimal data and bounded execution. Test effects, not just replies.Read →
Verification

2026-10-01 · 5 min · Verification

Telegram Bot Acceptance Checklist: 7 Tests Before Final HandoverSeven bot acceptance tests with a reusable sign-off table: ownership, duplicate actions, recovery, permissions and delivery evidence.Read →
Web

2026-10-01 · 5 min · Web

Website Traffic but No Leads: Find the Drop-Off Before Form SubmissionFind where website visitors stop before submitting an enquiry: intent, offer, CTA, mobile usability and form friction, with an evidence worksheet.Read →
Automation

2026-09-10 · 4 min · Automation

Recovering an abandoned Telegram bot: ownership, data and handoverRecover an abandoned Telegram bot by establishing ownership, preserving a restorable data snapshot and testing a controlled handover.Read →
Web

2026-09-09 · 4 min · Web

White screen in a Telegram Mini App: six checks before changing the setupDebug a blank Telegram Mini App with six evidence-led checks across launch mode, framing, storage, authentication, viewport and TLS.Read →
Web

2026-09-07 · 4 min · Web

Structured data in 2026: Google eligibility and four JSON-LD patternsUse structured data that matches visible content, separate Schema.org validity from Google eligibility and validate four illustrative JSON-LD patterns.Read →
Web

2026-09-06 · 3 min · Web

Fake crawlers: verify identity before changing nginx rulesSeparate claimed crawler names, verified provider identity and suspicious behaviour before tuning nginx access or rate limits.Read →
Verification

2026-09-05 · 3 min · Verification

Why AI agents game their own tests, and how to isolate verificationSeparate generated work from acceptance authority, protect the runner and allow a blocked result when requirements conflict.Read →
Automation

2026-09-05 · 4 min · Automation

A macro dashboard built by a multi-agent ensemble on DuckDBSeparate revenue breadth from its change, keep filing facts in DuckDB and check a multi-model synthesis against the source packet.Read →
Web

2026-09-05 · 4 min · Web

A high PageSpeed score versus real speed: what to checkCompare Lighthouse diagnostics with available field data and test real interactions before accepting a performance report.Read →
Automation

2026-09-05 · 5 min · Automation

Multi-dimensional portfolio stress-testing against liquidity shocksUse an explicitly scoped scenario grid and source/schema checks to inspect combined price, volatility and margin assumptions.Read →
Automation

2026-09-05 · 4 min · Automation

PWA and Telegram Mini Apps: mobile without the app storesA catalog, a booking flow, or a request form rarely needs an app-store cycle. One codebase can live as a PWA and a Telegram Mini App.Read →
Verification

2026-09-05 · 5 min · Verification

Silent no-ops and fail-plausible: when the system reports success and does nothingGreen dashboards and exit 0 are claims. Silent no-ops freeze data; fail-plausible agents invent a success report. How operators catch both.Read →
Verification

2026-09-04 · 3 min · Verification

How to verify an LLM answer: a verification layer that catches hallucinationsVerify material claims against retrieved evidence, check calculations with tools and keep uncertain answers out of automatic commitments.Read →
Reliability

2026-09-04 · 4 min · Reliability

Why leads from your website never arriveTrace a missing website lead from browser submission through persistence, transport and receipt before choosing a repair.Read →
Reliability

2026-08-31 · 5 min · Reliability

The chatbot went silent: debugging failures that produce no errorProcess up, logs clean, bot silent. Messenger bots fail without an exception. An external ping notices before a customer has to.Read →
Local models

2026-08-23 · 4 min · Local models

Local models vs API: compare workload, operating cost and data residencyCompare owned hardware, a rented GPU and a hosted API using measured workload, operating effort and a complete data-flow map.Read →
Local models

2026-08-23 · 6 min · Local models

What a local model can actually do: 75 business tasks, measuredTwo open-weight models on an office GPU, 75 office tasks, a code grader. Where a local assistant holds, where it invents, and what one rule changes.Read →
Automation

2026-08-21 · 3 min · Automation

An appointment bot that talks to the CRM in real timeDesign a booking assistant around the scheduling system of record, conflict handling, honest confirmations and a human handoff.Read →
Web

2026-08-21 · 6 min · Web

What actually moves rankingsCheck Google ranking guidance, relevance and technical access without fixed signal weights, automatic penalty claims or promises of position.Read →
Web

2026-08-20 · 6 min · Web

Usability audit: what it finds and what it's worthA usability audit checks whether visitors can complete the intended task, with findings tied to a page, device, evidence and proposed repair.Read →
Reliability

2026-07-01 · 3 min · Reliability

AI Agent Drift: The Five Failure Layers, DiagnosedDiagnose agent failures across task intent, conventions, environment, verification and carried context before choosing a remedy.Read →
Reliability

2026-07-01 · 3 min · Reliability

AI Agent Forgets Context Between Sessions — Why & the FixCarry decisions and blockers across agent sessions with a small, source-linked handoff; check the tool memory that is actually loaded.Read →
Reliability

2026-07-01 · 3 min · Reliability

AI Agent Infinite Loop: Fix the Death Spiral — Diagnose ItDiagnose repeated agent attempts with evidence of progress, bounded retries and an explicit stop-and-escalate path.Read →
Verification

2026-07-01 · 3 min · Verification

AI Agent Says "Done" But Nothing Changed: Verify the OutcomeCheck completion claims against the actual artifact, separate unknown outcomes and build a gate around the task result.Read →
Reliability

2026-07-01 · 3 min · Reliability

How Big Should a .cursorrules / CLAUDE.md File Be?Measure instruction tokens and task relevance; test loading and compliance instead of trusting universal line-count limits.Read →
Verification

2026-07-01 · 3 min · Verification

The Verification Gate Your AI Agent Is SkippingBuild a reproducible check entry point and verify its coverage, failure handling and connection to the actual task outcome.Read →
Reliability

2026-06-30 · 3 min · Reliability

Why an AI agent works one day and not the next: drift at the harness levelAn agent that worked yesterday can quietly do the wrong thing today. That is harness-level drift: the world moved, the instruction did not.Read →
Web

2026-06-26 · 3 min · Web

Website chatbot: template or customChoose a ready-made or custom website chatbot by its demonstrated grounding, integration, data handling and recovery capabilities.Read →
Reliability

2026-05-31 · 5 min · Reliability

AI, agentic system, autonomous agent: three levels of automation for an ownerAn owner who asks for AI may mean a chatbot, a scripted fleet, or a digital employee. Three levels, and which one to start with.Read →
Agent Reliability Lab
AI consultant

AI replies may be wrong. Messages and chat history are sent to an external AI service. Do not share passwords or confidential information. Privacy · Contact the studio directly