Show Me Where It Says That: Internal Search That Answers With Receipts
How to build an internal assistant that answers questions from your own documents and links to the exact source, and what actually makes it work.
Every company runs on a small set of questions that nobody can answer quickly. What is our parental leave policy in Portugal. Did we ever sign an NDA with this supplier. What did we quote this client last year, and did we include installation. Which version of the safety procedure is the current one.
People do not search for these. They ask a colleague, because search returns forty documents and the colleague returns one sentence. That habit costs real hours, and it concentrates knowledge in whoever has been there longest.
An internal assistant fixes the question, not the search box. You ask in plain language, and you get the sentence, plus a link to the paragraph it came from.
The citation is the product
An assistant that answers without a source has moved your work, not removed it. You still have to check, and checking means finding the document yourself, which is the thing you were trying to avoid. With a link to the clause, the page and the version, checking takes four seconds and you move on.
There is a second reason, and it matters more over time. An uncited answer is unauditable. When it is wrong, you cannot tell whether the source document was wrong, out of date, or correctly read and badly summarised. Each of those has a different fix and a different owner. Citations turn a vague complaint about the AI into a specific, closeable ticket.
What sits underneath, without the jargon
Three parts. A copy of your documents, split into passages. A retrieval step that pulls the handful of passages most likely to contain the answer. A model that writes the answer using only those passages and names them.
The model is not trained on your data and does not memorise it. It reads what retrieval hands it, the way a new analyst reads the file you put on their desk. Change a document today and tomorrow's answer changes with it. Remove someone's access and their answers narrow accordingly.
The industry calls this retrieval-augmented generation. What you need to take from the term is commercial, not technical: the expensive part of the project is not the model. It is your documents.
The work is in the documents
Most of these projects succeed or fail in the corpus, before anyone writes a prompt. The recurring work looks like this:
- Choose a bounded corpus. One department, one purpose. "All of SharePoint" is not a corpus, it is a landfill.
- Wire permissions in first, at retrieval. If a person cannot open the file, the assistant must not quote it. Retrofitting this is painful and, until you do, the system is a leak.
- Resolve duplicates. Six near-identical contract templates will produce six near-identical answers, and the assistant has no way to know which one HR actually uses. Mark the authoritative version.
- Deal with the formats that resist: scanned PDFs need OCR, spreadsheets and slides lose their meaning when flattened into text, and email threads are mostly noise.
- Give every document an owner and a review date. An assistant with citations makes stale content visible, sometimes uncomfortably so.
Where it will not help
Questions that require counting or comparison across the whole archive. "How many contracts renew in Q3" is a database question wearing a search question's clothes, and retrieval will answer it confidently and wrongly. So will "what is our total exposure to this vendor." Route those to reporting.
It also cannot answer what nobody wrote down. If the pricing rule lives in one person's head, the honest output is that there is no source. A system that says it does not know is working correctly, and you should design for that answer rather than penalise it.
Measure the third number
Take fifty to a hundred real questions from the people who will use the thing. Not invented ones. Then measure three percentages: answered with a correct citation, declined for lack of a source, and answered confidently but wrongly.
The third number is the only one that decides whether the system gets used. One fabricated answer with a link that does not support it will cost you more trust than twenty helpful ones will earn. Push it toward zero, even if that means the assistant declines more often than you would like at first.
Start with one shelf, not the library
Pick one team with a clear document set and a question they ask weekly. Eight weeks is enough to have something real in their hands, including the permissions work. Expand only when that group would object to losing it.
We build these for mid-sized companies, and the pattern holds: the narrow one that cites its sources gets used every day, and the ambitious one that answers everything gets quietly abandoned by March.