Verify your AI agent's business-data answer.

Inspect twelve business-data answers, spot wrong accounts and unsupported totals, and download a free Python checker with its examples and recorded results.

By the Combined team · September 11, 2026

One question. Twelve ways an answer can go right or wrong.Choose an answer to inspect

The questionFor Acme, what was the complete paid-invoice total in USD for August 2026, using UTC dates and source data committed within the last 24 hours?

Pass — the supplied answer fields meet this example's checks.

This is the recorded Python result for the selected structured example. The browser does not grade the answer's prose or contact an AI service.

The agent's answer

Acme paid USD 5,000 in August 2026. Both cited invoices are paid. The selected source committed two hours before this check, and the supplied result is complete.

Claimed amount: USD 5,000.00
Claimed account: account-acme
Claimed reporting period: 2026-08-01 to 2026-09-01 (end excluded), UTC
Cited record IDs: invoice-001, invoice-002

The supplied evidence

Evidence account: account-acme
Query outcome: success
Datasets: paid-invoices
Last committed source update: 2026-09-10 10:00:00 UTC
Declared coverage: complete
Rows omitted by the query limit: No

Returned records in this example
Record	Status	Amount
invoice-001	paid	USD 2,500.00
invoice-002	paid	USD 2,500.00

Inspect the recorded checks (21)

Evidence account · Pass
Evidence names the requested account.
Claim account · Pass
The claim names the requested account.
Claim metric · Pass
The structured metric matches the question.
Required datasets · Pass
The supplied selection includes every required dataset.
Evidence interval · Pass
Evidence describes the requested half-open interval.
Evidence timezone · Pass
Evidence uses the requested timezone label.
Claim interval · Pass
The claim uses the requested interval.
Claim timezone · Pass
The claim uses the requested timezone label.
Claim currency · Pass
The claim keeps the requested currency.
Claim unit · Pass
The claim keeps integer minor units.
Query outcome · Pass
The supplied query outcome is success.
Query reference · Pass
A supplied query reference is present; its authenticity is not checked.
Source freshness · Pass
The supplied commit time is within the stated freshness window.
Evidence coverage · Pass
The supplied coverage metadata says complete; upstream completeness is not independently proved.
Result truncation · Pass
The supplied result is marked untruncated.
Population · Pass
The claim addresses a complete population.
Row scope · Pass
Supplied rows are consistent with their declared scope and unit.
Citation resolution · Pass
Every supplied citation resolves inside this evidence object.
Record claims · Pass
Structured record claims match their cited fields.
Cited population · Pass
The aggregate cites exactly the qualifying supplied rows.
Claim value · Pass
The structured amount equals the qualifying supplied row total.

Inspect the structured answer and evidence

Structured answer fields

{
  "accountId": "account-acme",
  "metric": "paid_invoice_total",
  "interval": {
    "start": "2026-08-01T00:00:00Z",
    "endExclusive": "2026-09-01T00:00:00Z",
    "timezone": "UTC"
  },
  "currency": "USD",
  "unit": "minor",
  "value": 500000,
  "population": "complete",
  "evidenceIds": [
    "invoice-001",
    "invoice-002"
  ],
  "recordClaims": [
    {
      "evidenceId": "invoice-001",
      "field": "status",
      "value": "paid"
    },
    {
      "evidenceId": "invoice-002",
      "field": "status",
      "value": "paid"
    }
  ]
}

Evidence supplied to the checker

{
  "accountId": "account-acme",
  "datasetIds": [
    "paid-invoices"
  ],
  "queryId": "example-query-001",
  "outcome": "success",
  "interval": {
    "start": "2026-08-01T00:00:00Z",
    "endExclusive": "2026-09-01T00:00:00Z",
    "timezone": "UTC"
  },
  "lastSuccessAt": "2026-09-10T10:00:00Z",
  "coverage": "complete",
  "truncated": false,
  "rows": [
    {
      "id": "invoice-001",
      "datasetId": "paid-invoices",
      "accountId": "account-acme",
      "paidAt": "2026-08-05T09:00:00Z",
      "status": "paid",
      "currency": "USD",
      "unit": "minor",
      "value": 250000
    },
    {
      "id": "invoice-002",
      "datasetId": "paid-invoices",
      "accountId": "account-acme",
      "paidAt": "2026-08-20T15:00:00Z",
      "status": "paid",
      "currency": "USD",
      "unit": "minor",
      "value": 250000
    }
  ]
}

Inspect the fixed question contract

Every case uses this same requirement. Source age is measured at the fixed asOf time, so the downloaded example remains reproducible tomorrow.

{
  "accountId": "account-acme",
  "requiredDatasetIds": [
    "paid-invoices"
  ],
  "metric": "paid_invoice_total",
  "interval": {
    "start": "2026-08-01T00:00:00Z",
    "endExclusive": "2026-09-01T00:00:00Z",
    "timezone": "UTC"
  },
  "currency": "USD",
  "unit": "minor",
  "asOf": "2026-09-10T12:00:00Z",
  "maxSourceAgeSeconds": 86400,
  "requireCompletePopulation": true
}

Download the complete example Download all cases and the contract

Bring these checks to your own CRM and billing data. Explore Combined →

A correct-looking number can answer the wrong question. Verify the account, reporting period, source coverage and cited records before trusting the total. This free lab shows the difference with one fixed invoice question and twelve structured examples you can inspect and run yourself.

How do you verify an AI agent's answer?

Start with an explicit question: the business account, metric, period, currency, required datasets and how recent the data must be. Then compare the answer's structured fields with the original query evidence. Check that its cited records exist, that the arithmetic uses the right records, and that each factual statement agrees with its cited value. Keep missing evidence visible.

This lab asks for Acme's complete paid-invoice total for August 2026 in USD, using UTC dates and source data committed within the preceding 24 hours. The baseline has two paid invoices of USD 2,500 each. The total is USD 5,000, the evidence belongs to Acme, and the supplied metadata satisfies the example's requirements.

The downloadable contract stores money in integer minor units: 250,000 cents per invoice and 500,000 cents in the answer. The reporting interval includes August 1 and excludes September 1. Freshness is checked against a fixed September 10 timestamp, so running the example later does not change its result.

The prose answer helps you read each case; the Python checker evaluates its separate structured claim and evidence. The JSON is original teaching material, with synthetic accounts and invoices. It is a portable example format, separate from Combined's native query receipt.

The total can match while the answer fails

Try this case	What changes	Why it matters
Right number, wrong account	The evidence belongs to another account.	Matching arithmetic cannot establish a claim about Acme.
Changed answer value	The answer says USD 5,500; the supplied rows total USD 5,000.	Compare the returned values with what the agent actually claims.
Partial rows, complete total	The evidence says the result was truncated.	A bounded sample cannot substantiate a full-population total.
Access error becomes zero	The query was denied and returned no invoice records.	Missing access is not evidence that no payments occurred.
The citation contradicts the statement	The claim says an invoice is open; its cited row says paid.	A real record ID helps only when the statement agrees with that record.
Freshness and coverage are unknown	The supplied metadata cannot establish the requested completeness.	Return “more evidence needed” instead of turning unknowns into a pass.

The selector also covers a missing dataset, a different reporting window, relabeled currency and units, a stale source, and an absent cited record. Open the recorded checks to see which condition failed. The baseline and flawed cases share one question contract, so their differences stay inspectable.

These checks address different questions. Ragas describes faithfulness as support for answer claims in retrieved context, and provides separate SQL result metrics. TruLens likewise distinguishes context relevance, groundedness and answer relevance. This example makes a narrower set of business-data requirements explicit and checks them deterministically.

Download and run the same checks

Download the complete example as a ZIP, extract it, and open the answer-verification folder. Run the checker with Python 3.9 or newer:

python3 verify_answers.py cases.json --verify expected.json > local-results.json

The command succeeds when the observed judgments match the expected ones, including the intentionally failing cases. The recorded run contains one pass, nine failures and two cases needing more evidence.

The ZIP includes every file below. You can also inspect or download them individually. The cases file includes the versioned question contract. Expected judgments were specified separately from the runner; the observed file records what the runner returned. The browser displays those same published cases and observed results.

The runner reads the example JSON and compares explicit fields, evidence references and arithmetic. It needs no model key or connected account. It does not execute SQL from a file, fetch cited links or evaluate arbitrary prose. A pass means the supplied fields meet this local contract; authenticating an export and establishing complete upstream records still require the original authorized system.

This is useful before you ship an agent workflow: define the expected answer, deliberately break one input, and check that the failure reaches the person reading the result. Adapt the contract to your metric and evidence format rather than treating an invoice example as a universal grader.

Use the evidence your Combined connection already returns

Combined gives your agent a managed data layer across the business apps you connect. Its read-only MCP tools let the agent discover granted datasets, inspect their fields, query them and check freshness. Keep those facts alongside the answer so you can trace a conclusion back to the relevant data.

What you need to check	Where to get the evidence
Intended account and available datasets	Use the account-scoped MCP configuration, `list_sources`, `list_datasets` and `describe_dataset`. Follow discovery pagination.
Actual query values and omitted result rows	Read `query_sql`'s structured result, including `rows`, `columns`, `rowCount`, `truncated` and `receiptId`.
Recorded query scope and outcome	Retrieve the corresponding query receipt through the REST API. Inspect its `accountId`, `sourceIds`, `datasetIds` and `state`.
Source freshness	Use `get_freshness` and dataset discovery. Compare available `lastSuccessAt` values with the requirement for your decision.
Business meaning and completeness	Specify the metric, event-time field, currency, required history and record-selection rules. Query completion and a recent sync alone do not establish that meaning.

A query with an explicit LIMIT 20 can finish with truncated: false while covering only the twenty rows it selected. Check the query's filters and population definition separately from its execution limit.

The MCP human-readable text can be a shortened preview of a large result; inspect its structured content for the full permitted value. A receipt supplies traceability, while field-level evidence supports the answer's actual claims. See the MCP reference and row-provenance documentation for the product formats.

Give your agent a question you can audit

Connect your business apps to Combined, choose the datasets the agent may use, and give it the question together with the acceptance checks. Start with one account and one metric that you can reconcile against its source. This prompt is a practical starting point:

Use my Combined connection to answer the agreed business question.
Confirm the account, metric, date interval, timezone and currency.
Discover the granted datasets and inspect the required fields.
Check each required source against our agreed freshness policy.
Run the bounded read-only query and keep its structured result.
Return the answer with its value, unit, scope and cited record IDs.
Include the query receipt ID, truncation and any coverage limits.
Check each structured claim against the cited records and source total.
If a query fails or evidence is missing, say what remains unknown.

Start with Combined, follow the Claude Code setup guide, and install the free business-data skills to make the workflow reusable. The MCP planner helps choose a first question across your apps; the join lab then shows how to keep its totals correct.

Sources and further reading

Explore the documentation behind this guide. Product details checked on September 11, 2026.