Chapter 10 of 26

One stale number can poison the whole session

Say we are testing credit cards at Zinc Bank, the practice bank we use in class, with an agent that will read the requirements, write the test cases and automate them. A test case is one check: a new customer with a credit score around 700 to 730 should get a certain credit limit. Automating it means turning it into a script that runs by itself and compares the bank's answer with the expected one.

You do the first part right. You hand the agent only the current requirement, which says that customer's limit is $15,000.

Overnight it logs twelve defects in Jira, the tracker where bugs go to the developers. Every test expected $5,000, and the bank said $15,000. Fail, fail, fail. Your developer opens the twelve tickets and gets angry, because the bank's software is right. The limit really is $15,000 now.

You never gave the agent $5,000. So where did $5,000 come from?

It went reading on its own. While it worked, it opened an old folder and picked up a test result from sprint two, a year ago, when the limit for that customer was $5,000. Once it had that number, everything after was built on it. That is context poisoning. Your entire session is based on the wrong context, and everything it does is trash.

Do not ask the agent where it got the number. The session record shows every file it opened, and the old test result is right there. Twelve defects, one cause, and the cause is in the context, not the code.

Then write the fix into the rules file the agent reads at launch. Some tools call that file CLAUDE.md, others AGENTS.md. The rule is a whitelist, not a judgment: only the documents I give you are your context, and if you pick up an old requirement, an old execution log or an old bug report on the way, ignore it. Do not get obsessed with it. The agent does not have to know which page is old. It only has to know which pages are yours.