AI & Automation
Also known as
Golden dataset · Reference set
Definition
Related terms
Benchmark
A fixed, documented test used to compare systems on the same task and data. Vendor-run benchmarks on small or self-selected samples are observations, not laws — the questions that matter are sample size, data provenance, and whether the evaluation set is available for independent replication.
Precision and Recall
Precision is the share of returned results that are correct; recall is the share of correct results that were returned. The F1 score is their harmonic mean. In finance operations the two carry different costs — a false positive wastes review time, a false negative may pay a fraudulent invoice.
Model Drift
Degradation of accuracy over time as inputs shift away from what a system was tuned on — new supplier formats, changed remittance layouts, revised chart of accounts. Drift is silent by default, which is why continuous measurement against ground truth is an operating requirement, not a project phase.
More in
AI & Automation
Abstention
A system's ability to decline to answer when it lacks sufficient grounds, rather than producing its best guess. In high-stakes finance work, a well-calibrated system that abstains reliably outperforms a nominally more accurate one that always answers, because the cost of a confident wrong answer is not symmetric with the cost of a flagged one.
Agentic AI
Software that pursues a goal across multiple steps — reading context, calling systems, making decisions, and acting — rather than producing a single output on request. In finance operations the distinction that matters is not autonomy but accountability: an agent that acts on the ledger must produce evidence a reviewer can follow.
Agentic Search
Retrieval that plans and executes several dependent queries to answer a question no single lookup can resolve — for example tracing a disputed charge across a contract, a purchase order, three invoices, and a remittance file. Evaluated on end-to-end answer correctness rather than on document relevance.
Confidence Score
A model's self-assessed reliability for a specific output, used to route work between automatic processing and human review. The score is only useful if it is calibrated: a system confident at 95% must be right about 95% of the time at that level, or the threshold is a fiction.
Exception Queue
The holding area for transactions that could not be processed automatically, awaiting human resolution. Queue design — prioritization, evidence pre-assembly, routing to the right skill — determines whether exceptions are cleared in minutes or days, and is where most of the realized value of automation actually sits.
Grounding
Tying every generated assertion to a specific source passage a reviewer can open and verify. Grounding is the difference between an output that can be audited and one that can only be trusted, which makes it a compliance requirement rather than a quality nicety in regulated finance work.
Low-risk start
Watch it run on your real documents and your own systems, before you commit to anything.