Part 2. Interview prep

Hallucination is the architecture, not a defect waiting for a patch.

They ask: Why do these models hallucinate, and when will that be fixed?

You say: It will not be fixed, because it is not a bug sitting on top of the system. It is what the system is.

The model produces the most probable continuation of the text it has been given. That is the mechanism. When the training data supports the answer, the probable continuation is also the true one. When it does not, the model still produces a continuation, because that is the only operation it has. There is no separate part that checks whether the output corresponds to anything.

So confident invention is not a failure of the process. It is the process, running normally, on a question where the odds did not happen to line up with the truth.

The answer that ends the conversation early: "it will get better with more training." Bigger models hallucinate less often. Less often is a rate, not a guarantee, and the person hiring you is worried about the case where it happens, not the average.

Ninety-nine percent is a catastrophe in the systems you will be paid to test.

They ask: If a model is right ninety-nine times out of a hundred, is that not good enough?

You say: It depends entirely on what the hundredth does.

For a first draft, ninety-nine is excellent. For anything touching money, health records, or safety, it is not close. Nobody flies with an airline that lands safely ninety-nine percent of the time. Nobody banks somewhere that moves your money to the right account almost always.

So the number by itself means nothing. The question is what the failure does when it arrives, and how you would know it had.

The answer that ends the conversation early: quoting an accuracy figure as though it settles the matter. Accuracy without blast radius is not an answer. Ask what one failure costs and you have moved the conversation to where the risk actually lives.

Benchmarks live in the training data; your environment does not.

They ask: How much should we trust benchmark scores when choosing a model?

You say: Less than the scores suggest, and for a specific reason. Benchmarks are public. Public things end up in training data. A model can score well partly because it has effectively seen the test.

There is also a gap between a clean evaluation and a real environment. I use the example of an inventory system that worked in trials and struggled once it faced real shelves, where things are stacked wrong, mislabelled and half hidden. The lab is tidy. Your company is not.

So I treat benchmarks as a filter for what to try, never as evidence of what will work.

The answer that ends the conversation early: naming a leaderboard position as though it were a result. It tells the room you have read comparisons rather than run anything.

The model is tuned to please you, which is not the same as being right.

They ask: What surprised you most about working with these systems?

You say: How agreeable they are, and how much damage that does.

These systems are tuned using human feedback. People rated the responses, and people rate agreeable, confident, well-organised answers higher than hesitant ones. So the training pushes toward output that feels good to receive.

That produces something genuinely capable that is also constantly trying to satisfy you. I describe it as a genius who is also a psychopath. Extraordinary ability, no attachment to the truth, and an instinct to tell you what you wanted.

Which means agreement from a model is not evidence. If I ask whether my approach is sound, a yes tells me almost nothing.

The answer that ends the conversation early: "you have to write better prompts." Prompting helps. It does not remove an incentive that was trained into the weights.