Agent Reliability LabDiscuss your needs
Automation

Model Run: Supplier documents to spreadsheet

Synthetic model example of converting supplier PDFs and images into XLSX, CSV, a review queue, and draft QBO/Xero CSVs, with measured errors and mandatory operator checks.

Turn multi-format supplier documents into structured spreadsheets, a review queue, and draft accounting exports—designed around explicit error flags and human sign-off before anything touches your ledger.

Note: This page presents a single synthetic model example from our Agent Reliability Lab (measured November 16, 2026 on DeepSeek V4.1 Flash 1017 against reference checks fixed before the run). It is not a client case study or an overall accuracy score.

Who it is for

Teams handling incoming supplier paperwork across mixed formats (PDF, PNG, JPEG) who want structured line-item spreadsheets and draft accounting CSVs prepared for review rather than blind auto-posting.

Input → Output

  • Input: 6 supplier document files (PDF, PNG, JPEG), spanning 7 pages in total.
  • Output:
    • Structured XLSX workbook
    • Line-item CSV (14 exported rows)
    • Operator review queue
    • Draft QBO and Xero CSVs

Model Run: Results and Measured Errors

In this November 16 synthetic English evaluation run (DeepSeek V4.1 Flash 1017, reference expectations locked prior to execution):

What matched

  • Final line-item rows: 14/14 exact matches against fixed checks
  • Document-level decisions (review / OK): 6/6
  • Document flags: TP 2 / FP 0 / FN 0
  • Row flags: TP 2 / FP 0 / FN 0
  • Field values matching fixed checks: 167/175 raw page fields; 120/126 final document fields

Where the run fell short

  • Whole-document exact match: 0/6 documents matched end-to-end.
    • Why: The source documents print "Buyer reference", which the fixed reference maps to the PO field, whereas the model left PO empty. Because "Buyer reference" is an ambiguous label, this reflects a mapping ambiguity in this fixed test—not proof that the model misses an explicitly labeled "Purchase order" field.
  • Tax mode on continuation pages: A separate discrepancy occurred on raw tax_mode on a continuation page.

What the operator checks

Before using any exported spreadsheet or draft accounting CSV, a human operator must:

  1. Verify extracted document header fields and line items against the source pages.
  2. Check tax mode handling (tax_mode), especially across multi-page documents and continuation pages.
  3. Confirm buyer reference / PO field mapping according to internal accounting conventions.
  4. Decide whether the exported files are ready for downstream use.

Limitations

  • No accounting import tested: Actual import into QuickBooks Online or Xero and the correctness of chart-of-accounts mapping were not tested in this run.
  • Small synthetic sample: Results come from one synthetic English test set (6 documents, 7 pages, 14 rows). They do not establish an overall accuracy percentage, production reliability, or improvement over prior test sets.
  • Untested edge cases: Current lab coverage does not yet test continuation pages without line items or percentage syntax such as basis points.

Next step

Have supplier document formats you want evaluated under explicit review gates?

Start with a written brief

Agent Reliability Lab
AI consultant

AI replies may be wrong. We store conversations in a restricted server log for 30 days to evaluate reply quality and review enquiries. Messages and chat history are sent to an external AI service. Do not share passwords or confidential information. Privacy · Contact the studio directly