- citations
- compliance
- RAG
How to verify an AI answer about your documents
An answer without a source is not an answer. It is a hypothesis. A practical guide to checking what an AI system claims about your own documents.
When a legal, compliance, or research team adopts AI over its own documents, the question that matters is not how good the answer sounds. It is whether someone can check it before signing anything.
What makes a citation verifiable
A verifiable citation has three parts. Drop one and verification breaks.
The document
The name of the file the claim came from. Not "your documents" or "applicable regulation", but the specific file.
The location
The page, clause, or section. A 400-page PDF with no page number is not a source; it is a search you still have to run.
The passage
The literal text supporting the claim, so you can compare it against what the system said it means.
With those three, verification takes seconds. Without them, the checking work is identical to having done the search by hand.
High-risk AI systems shall be designed and developed in such a way that their operation is sufficiently transparent to enable deployers to interpret a system's output and use it appropriately.
That is what a complete citation looks like: document, page, and passage. A reader can open page 46 and confirm it without asking anyone.
Why models invent sources
A language model predicts probable text. Ask it about a contract without giving it the contract, and it will produce something answer-shaped: a plausible clause, a reasonable page number, a confident tone.
The difference between a system that cites and one that invents is not the model. It is whether the answer is built from passages retrieved out of your documents or from the model's memory.
| Model without documents | Model with retrieval | |
|---|---|---|
| Source of the answer | Training memory | Passages from your files |
| Citation given | Plausible, sometimes nonexistent | The passage actually used |
| How you verify | Manual search | Open the cited page |
| How it fails | Silently | Visibly: it finds nothing |
The last row is the important one. A good system fails by saying "I could not find this in your documents." A bad system fails by answering confidently.
A three-step verification protocol
For any claim that will support a decision:
- Open the cited source. Not the system's summary of it, but the original document, at the page given.
- Compare the literal passage. Does it say what the system claims, or something adjacent?
- Ask for the opposite. If the system says something is permitted, ask which documents suggest otherwise. An honest system surfaces the exceptions.
Step three is the one most people skip and the one that catches the most errors.
3
parts per citation
document, page, passage
<10s
to verify
when the citation is complete
80+
file formats
supported in BrainBox
What to demand from a vendor
Before buying, ask for a trial on your documents, not the demo set:
- Upload a document you know thoroughly.
- Ask a question whose answer lives on exactly one page.
- Check that the citation points to that page.
- Ask a question whose answer is not in the document at all.
That last question is the real test. If the system answers just as confidently as before, it is not reading your documents.
Frequently asked questions
- What is a verifiable citation in an AI system?
- A reference that includes the document name, the exact location inside it (page or section), and the literal passage supporting the claim. A filename alone is not a verifiable citation, because it lets you check nothing.
- Why do language models invent sources?
- A model generates the most probable text given its context. If the documents are never given to it as context, it produces something shaped like a citation, with the right format and a plausible page number, and no real passage behind it.
- How long should verifying an answer take?
- Under ten seconds per claim. If verification costs more than looking the information up by hand, the system is not saving work. It is relocating it.
Written by
BrainBox Team
Document intelligence, by ExaByte Company
We build BrainBox — the platform teams use to ask questions across their own documents and get answers with exact page citations.