Nightly price monitoring for a regional supplier
An agent that scrapes six competitor sites every night, matches SKUs across messy names, and delivers a comparison the operator can read in minutes.
What changed. Six competitor sites are compared daily; acceptance requires dated source observations and explicit missing or unmatched rows.
Stack. Python · Playwright · LLM matching · scheduled reports
Context
A regional construction-materials supplier had to know where it sat on price against six competing storefronts. The work was manual: open each site, search by name and spec, paste into a spreadsheet, and hope the SKUs actually matched. Names and pack sizes differed across stores. Manual copying delayed comparison. By the time the sheet was finished, some of the prices were already stale.
Task
Replace the weekly scavenger hunt with a loop that runs unattended, matches products even when titles disagree, and delivers a structured comparison the commercial team can act on the same morning. The agent must not invent a match. Anomalies — missing pages, sudden price spikes, unmatched SKUs — have to surface as flags, not as silent blanks in a green report.
What we built
A nightly agent that visits the six competitor sites, extracts prices and attributes, and writes a CSV/Excel comparison. Playwright drives the storefronts that will not answer a plain HTTP request. A language model proposes fuzzy matches across divergent titles and pack formats; the report still lists unmatched rows instead of forcing a join.
The run is scheduled. Check freshness per source and collection attempt: if a scrape fails, preserve the observation time and mark its rows stale or unavailable. A newly written report file does not make old observations current. The operator reads where the house is cheaper, where it is more expensive, and how the spread moved over the week. Pricing decisions stay with a person.
What changed
The daily schedule sets a target, not a freshness guarantee. Failed collections must remain visible; an incomplete run must not be presented as a complete daily comparison. No measured time-saving claim is made here. Six competitors stay on the same sheet, with names reconciled where the matcher is confident and left explicit where it is not.
What we would do differently
Treat each competitor parser as a versioned contract with a fail-loud layout check, rather than a script that “usually works.” Storefront HTML can change between runs. The first useful alarm is not a smarter matcher — it is a scrape that refuses to publish when the page shape changed and the extract came back empty.
Related service: AI agents and automation