July, a Saturday, near the end of the term.
I had asked the class to look at the statement export before the session. Download a transaction history from the banking application we use for practice, and come back with what they found.
Almost nobody had.
So I shared my screen and we did it together, and what I wanted them to see was not the defect. It was how much work the word verify was hiding.
We opened an account and asked for the statement. Straight away there was a problem. Pick an account with no transactions in the period and the application would not let you go forward. It just stopped. Not an error message explaining anything, and no sensible fallback to an account that did have history. Somebody had never considered an empty result, so the empty result had nowhere to go.
That took a minute to find and it is the sort of thing an agent will walk straight past, because nothing crashed.
Then we got a statement out, and I asked the class the real question. How do you check this document is correct.
Somebody said open it and look. So we looked. There was a logo. There was a name at the top, an account number, a date range. Then rows. Merchant, date, identifier, amount. Three hundred of them.
I asked them to say out loud what checking that actually means. Not the idea of checking. The work.
It means the logo is the right logo. The name matches the account holder and not somebody else. The date range matches what was requested. Then every row on that document has to match the row on the screen it came from. Merchant name, transaction identifier, amount, in the right order, with nothing missing in the middle and nothing invented at the end.
Three hundred rows, several fields each. Somebody worked out it was around nine hundred comparisons.
By hand, carefully, with a break because nobody's attention survives that intact, you are looking at three hours. And at the end of three hours you would have checked one statement, for one account, for one date range. Change the range and you start again.
That is when the room understood why we automate. Not because automation is modern. Because the manual version of this job cannot be done at the frequency the business needs it done.
Then I showed them what it looks like when a script does it. Read the rows off the screen, read the rows out of the document, compare them one to one, and report the ones that differ. Minutes.
Somebody asked whether you have to compare all three hundred every time.
I said no, and this is where judgement comes in. There is a method called equivalence class partitioning, which is a heavy name for something you already do at the market. You do not taste every grape in the basket. You taste one from the top, one from the middle, one from underneath, and if all three are sweet you buy the basket. If one is sour you look harder.
You take a sample from the beginning, the middle and the end, and if those match you accept the set. It is not a guess. It is a decision about where failures would show up if there were any.
But I told them the honest version too. Sampling is what you do when time is short, in a hot fix, when something is on fire and the fix has to ship. If you can afford to compare all nine hundred by machine, compare all nine hundred. The sample exists for when you cannot.
We spent the rest of the session on the part that actually matters, which is what an agent does with all of this. Because an agent will happily generate a script that compares the document to the screen. It will run it. It will tell you it passed.
And unless you know that the logo, the name, the date range and every row are the things that had to match, you have no way to tell whether its script checked any of them.
That is the job now. Not doing the comparison. Knowing what the comparison had to include, so you can tell whether the thing that did it for you actually did.