Metrics & Benchmarks
Also known as
Evaluation set
Definition
Related terms
Ground Truth
A verified, human-adjudicated set of correct answers used to measure system performance. Without a maintained ground-truth set, accuracy claims cannot be checked, and the absence of one is the most reliable tell that a vendor benchmark is not reproducible.
Extraction Accuracy
The share of extracted fields that are correct. The metric is only meaningful with its scope stated: header fields on clean PDFs, line items on scanned faxes, and full-document capture across a supplier long tail produce very different numbers from the same system.
Precision and Recall
Precision is the share of returned results that are correct; recall is the share of correct results that were returned. The F1 score is their harmonic mean. In finance operations the two carry different costs — a false positive wastes review time, a false negative may pay a fraudulent invoice.
More in
Metrics & Benchmarks
Auto-Containment Rate
The share of inbound requests or queries resolved end to end without human escalation. Borrowed from service operations and now the standard measure for agentic handling of supplier and customer queries, where it is a cleaner signal than volume handled.
Cell-Level F1
The harmonic mean of precision and recall computed per table cell, the standard measure for evaluating table extraction. More demanding than row-level or document-level accuracy because it penalizes both missed content and misplaced content.
Cost per Invoice
Fully loaded cost to process one supplier invoice from receipt to payment. Best-in-class operations run in the low single dollars; median performance sits around $10–$15. The spread is driven almost entirely by exception rate and approval latency rather than by capture technology.
Exception Rate
The share of transactions requiring human intervention before completion. Around 14% is the common industry benchmark for payables with best-in-class below 10%. The rate matters less than the mix: a small share of genuinely complex exceptions can consume more effort than a large share of trivial ones.
Extraction Accuracy
The share of extracted fields that are correct. The metric is only meaningful with its scope stated: header fields on clean PDFs, line items on scanned faxes, and full-document capture across a supplier long tail produce very different numbers from the same system.
Full-Time Equivalent (FTE)
A unit of staffing capacity equal to one full-time person. FTE is the historical pricing and benchmarking unit for finance operations, and the reason cost reduction in traditional outsourcing is bounded by how far headcount can fall.
Low-risk start
Watch it run on your real documents and your own systems, before you commit to anything.