They ask: How do you reduce invention in a system answering from company documents?
You say: You force it to cite, and you give it permission not to answer.
Citing means every claim comes back with the document, the section and the line it came from. That does two things. It gives you a way to check without reading everything. And it constrains the generation itself, because producing a specific reference is harder to do about something that does not exist.
The second part matters as much. You have to explicitly tell it that saying it does not have the answer is an acceptable response. Left alone, these systems will produce something, because producing something is what they do.
Giving a model permission to say it does not know is one of the few instructions that changes behaviour reliably. Without it you have asked a machine that always answers to sometimes not answer, and you have not told it that silence is allowed.
The answer that ends the conversation early: "we tell it to only use the provided documents." That instruction sits in a context window that will be full later. Requiring a citation gives you something you can check afterward.
They ask: Explain retrieval augmented generation to a manager who does not write code.
You say: Hire an outstanding chef. Genuinely excellent, trained for years. On the first morning they still cannot cook your menu, because they do not know your recipes, your suppliers, or where anything is kept.
The chef is not deficient. They are ungrounded in your kitchen.
Retrieval augmented generation is handing the model your recipes. You connect it to your own documents so that when it answers, it answers from your policies rather than from general knowledge.
Two things follow. The quality of the answers is now the quality of your documents. And a chef who has read your recipes can still improvise something that is not on the menu, which is why the checking does not stop.
The first of those two catches people out. Teams connect a model to their documentation and are disappointed by the answers, and the documentation was already wrong. The system did not degrade the content. It exposed it.
The answer that ends the conversation early: describing it as giving the model more data. It is not volume, it is provenance, and stakeholders understand provenance when you use a kitchen.
They ask: How do you find the defects that cause real damage?
You say: I ask one question during requirements. What can go wrong with this in production.
Not what should it do. What happens when it runs for years, at volume, with real people using it in ways nobody described.
The example I use is an identifier that was defined with a fixed number of digits. It worked for years. Then the system generated the last available number and every process that depended on it stopped, across a large number of institutions at once. Nothing was broken in the code. The design had assumed a ceiling would never be reached, and nobody had multiplied the numbers to check.
That question, asked once, at the right time, costs a meeting. Not asking it cost an enormous amount and it was found by the users.
I ask it now about anything with a limit in it. How many digits, how many rows, how many years. Most of the time the answer is fine. The cost of asking is a minute and the cost of not asking arrives all at once.
The answer that ends the conversation early: describing your process for writing test cases. They asked how you find serious defects. Serious defects are usually decided before any code is written.