Chapter 28 of 32

Part 6. The agentic testing lifecycle

The bad requirements document I wrote on purpose.

June, and the Saturdays after it.

I built the class a banking application with bugs in it.

Not by accident. I put them there. Money that goes missing on a transfer, a page that reports success and shows nothing, a form that accepts a double click and creates two transactions from one intention. Real defects of the kind that reach production and cause meetings.

Then I did something that got a better reaction than the application.

I wrote them a bad requirements document.

I sat down and deliberately produced the sort of specification people are actually handed. Vague where it should be exact. Silent on what happens when things are empty or absent. Sentences that sound like requirements and cannot be tested. The application should be user friendly. The user should be able to see their transactions.

Then I wrote the good version of the same document beside it, and I gave them both.

I told them the truth about why. When you join a company you are going to be handed a document like the first one, and there is a strong chance nobody in the room will tell you it is bad. Half the specifications I have been given in my career would not survive a careful reading. That is not a complaint about business analysts. It is what the job looks like.

What I wanted them to see is that the difference between the two documents is not writing quality. It is whether a sentence can fail. The user should be able to see their transactions cannot fail, because there is no state of the world that contradicts it. A registered account holder can retrieve statements for any complete month in the last two years, and an account with no activity in that month returns an empty statement rather than an error, can fail. You can go and check.

That distinction is the whole of requirements work and it takes about four minutes to teach.

Then we built the loop.

I walked them through what I actually wanted, which is not a person writing tests faster. It is a chain. An agent reads the requirement and its acceptance criteria. It produces test cases from those criteria, one per criterion, so that nothing is covered twice and nothing is missed. It generates the automation. It runs it. When something fails, it does not simply report red. It quarantines the failure and looks at it, because a failure is one of three things. The application is genuinely broken. The test was wrong. Or something moved in the interface and nothing is actually wrong at all.

Only the first of those is a defect. The chain has to be able to tell them apart, and when it cannot, it escalates to a person instead of guessing.

I told them the part that makes this real. If the same test fails once, that is noise. Three times identically, that is a defect, and now you have something worth a human's attention.

Somebody asked whether this means they should stop learning to write automation by hand.

I said the opposite, and I said it carefully, because it is the question underneath the whole course. Writing the automation yourself is now the manual step. That is a strange sentence and it is true. For years, automation was the thing that replaced manual work. Now there is a layer above it, and typing the script by hand is the slow, careful, human version of a job the agent will do in a minute.

But you cannot direct that agent if you have never written the thing yourself. You will not know what a good page object looks like, or why a test that shares state with another test is a lie, or what the failure means when it arrives. You will be a person forwarding output you cannot assess.

At the end somebody asked who decides when it goes to production.

I said that has not changed and I do not think it is going to. Local, then a testing environment, then production, and somebody with a name says go. The agent can prepare everything up to that line. It does not get to cross it.