The Agent Factory

I was twenty minutes into a Saturday class. Then I said $200,000, and everybody woke up.

Imagine I run a firm with ten accountants. Each one costs me $6,000 a month, $72,000 a year, for eight hours a day. That is $60,000 a month, $720,000 a year, for people who log into a browser and put numbers into boxes. An agent is a program you tell in plain English what you want; it opens the browser, does the job on its own, and keeps going until it is done. If one of my students builds me an agent that does that role, I would love to give that student $10,000 to build it. Not for the typing. For the $60,000 a month.

Old software automated the task. The new software automates the role, and the people who already automate the role in a test environment are the ones who get paid to build it.

The day before class I happened to see a job posting. It was about Playwright, but it was not about test automation. Playwright is the tool this class was starting that morning, a program that opens web pages, clicks the buttons and types into the forms by itself, the way a person would, and testing is what it is usually used for. I looked into what the company was going to use it for. It was a cybersecurity company using Playwright for agentic penetration testing: a program that clicks through your own website on its own, trying to break in before somebody else does. I had been using the tool for almost every kind of agentic automation and I had never thought of that as a thing. It was a thing.

The next morning I told the class about the job listing. There were no years of experience in it. It named the tools, it named large language models, the AI brains an agent runs on, it named agentic frameworks, and that was all it asked for. I told them what I saw in it: learn Playwright, sit down with this for a week or two, and I think there's nothing in that job description that you guys cannot do.

Then I asked them how much it paid.

$80,000 a year, one student said. I said okay, $80,000, what else, anyone else. Nothing. A room full of people, and nobody would say a number out loud.

So I told them. Come on, guys. Throwing numbers are free. You don't lose anything.

$100,000, she said. Go up a little bit, I said. $120,000. Go up. $150,000. No. Go up. It can't be more than $200,000, she said, and she stopped there.

She stopped at $200,000 a year. How far off was she?

$9,000. $209,000 a year, for a tool they were starting that morning. She was nearly right the moment she stopped being afraid of the number, and the first guess in the room was less than half of it. Not for the tools; the tools were two weeks of sitting down. For a penetration test that somebody is already willing to pay for, at somebody else's price. The tool we use to pretend to be a user in testing had been hired to be the user for real, and that is the number that woke the room.

Ask yourself what buying anything was before the website. Without Amazon, I get out with my child, I drive to Walmart, I sit in traffic, I park. I walk the store asking an employee where the section is, the product is sold out or it is not, I stand in the line, I pay, I drive back. Sometimes it takes four hours for the entire transaction. The website did not make the product. It took the four hours out.

Instacart is a good one. I used to go to Costco, and now I don't have to go. If your hourly rate is $50, a Saturday in Costco is four hours, $200 of your time, and you pay the guy who brings it $20 or $30. The gap is the value, and the company creates it for a hundred million people at once, at almost no extra cost. Uber did the same thing to the yellow cab. Software is process automation: faster, cheaper, better, at scale. If it does not do that, it is not good software.

That gap, times everybody, is the whole software business. In 2011 Marc Andreessen wrote in the Wall Street Journal that software was eating the world, and it did. Public software companies still keep about 71 cents of every dollar they take in, because once the app is written, the next customer costs almost nothing. And software was sold to the people who did the work. The work itself, the salaries and the outsourced services, is worth $4.6 trillion by one venture firm's 2024 count, and they named what the agents are pointed at: service as software.

The same business reprices the old job. A license to drive a yellow cab in New York, a medallion, a metal plate on the hood that says you may pick up passengers, sold for $1.3 million in 2014.

The task those drivers did was automated by an app. What is that license worth now?

$70,000 to $80,000 by 2021, from more than $1 million. Nothing happened to the driving. The task was automated, and the license was priced on the task, so the price went with it. Hold on to that, because it is exactly what the new software does, one level up.

Think of an ATM. Ten of us ask for $20, and the process is the same for all ten: insert the card, enter the PIN, ask for cash, get $20. That is the first level, deterministic software. Your banking app is the same: every customer sees the same screens in the same order.

The second level is the software that remembers you. Picture a restaurant. The first waiter has served you many times and still has to be told your name, that you have a peanut allergy, that there is a table you hate, and she makes the same mistakes every visit. The second waiter knows the moment you walk in, from the day of the week, whether this is business or a casual dinner, gives you the quiet corner when it is business, and already knows your appetite. The second experience is above and beyond. Amazon is still deterministic underneath, but it knows your address, and an address is a home price, and a home price is an income, so it shows you things in your bracket. From what you buy it knows whether you have a child and whether you are married, and it fills the suggestions from that. YouTube's recommendation engine is the same idea, hit or miss.

The third level is the agent. The second level knows you and shows you things; the third does the work itself. Instead of process automation, we automate the entire role. QuickBooks is a very fancy version of an Excel sheet. It is the program a small business keeps its money in, and an Excel sheet is a grid of numbers on a screen. Before it, a bookkeeper typed everything into that grid and added the calculations by hand. QuickBooks automated many of those processes, and one bookkeeper could do the work five or six used to do on the sheet. That is process automation, and it is where the old software stopped: it could do the process and not the role. Somebody still had to sit at the keyboard, read the statement, decide what each transaction was, and close the month.

In class I hooked an agent up to a bank application and told it, in one sentence, to get me the statement. What did it do first?

It logged in, the way the bookkeeper does. Then it downloaded the statement, pulled fifty-seven transactions out of it, and ran the analysis while I was still talking. It did the role, from the login onward. That is the third level: hook your system up, connect the agent to the bookkeeper's QuickBooks, and your agent does the job that was done by the bookkeeper, end to end.

For seven or eight years I had a CPA firm. $3,000 a month on retainer, $36,000 a year, and the corporate tax on top, $5,000 to $8,000 depending on the year. Last year the tax alone was $8,000. They were expensive, they had employees, and they had hundreds of clients.

This year I did the tax myself, with the agent. Not to save money. I was so mesmerized by what it could do that I wanted to see it run.

I went into QuickBooks and downloaded everything. The PDFs, the spreadsheets, the CSV files. Ten years of financial transactions, line by line, item by item. Every time somebody swiped a company card, every payroll, every trip. Tens of millions of lines. Then I let the agent audit every year's tax return against them.

It held all ten years in its memory at the same time and looked at them at once. Can any CPA do that? The best accountant in the world, can he see ten years of transactions simultaneously? It is biologically impossible.

Ten years of books, every return, one agent. What did it find?

Issues. Overpaid tax, more than one year of it. And then the number that hurt more than the tax. I had been paying these people thousands of dollars a month and getting one percent of their attention, because I was not their only client. I was one of hundreds. Now the agent could do all of that job for me. So why do I still have to pay them? I asked the class the same question.

Name a role you know, and it is already on the list.

The bookkeeper first. In July 2025 Intuit put a team of agents inside QuickBooks, and its accounting agent, in the company's own words, automates bookkeeping and transaction categorization and assists in reconciliation.

The paralegal reads contracts and finds the inconsistency in the document, and charges $50 or $60 an hour for it. Why would you pay $60 for the thing an agent does for free? A legal agent called Harvey reached $100 million a year in revenue in August 2025, and $400 million a year later. It is used inside half of the hundred largest law firms in the United States. One of its features reads ten thousand documents against a hundred questions in one pass.

In June 2025 a company called XBOW put a fully autonomous agent on HackerOne, the marketplace where companies pay hackers to find their holes. It became the first machine to reach number one on the United States leaderboard, above every human hacker on it, with about a thousand submitted vulnerabilities, each one checked by running the attack in an invisible browser.

The customer-service agent: in February 2024 Klarna said its assistant had held two point three million conversations in a month, the work of seven hundred full-time people, resolving in under two minutes what humans took eleven minutes to resolve.

The shopper: in April 2025 Visa and Mastercard let agents pay with your card, search, choose and pay on your behalf, and Amazon began testing an agent that buys for you. The cheapest ticket, the auction bid at lightning speed, the house with the most upside in a market you have not read: those are not my fantasies.

And the front desk. I am building one now for a dental office, where the person at the front desk costs $100,000 a year and is very hard to find.

In April 2026 Meta began recording its employees' keystrokes, mouse clicks and screens on a list of work apps. What for?

Not to grade them. To train agents. In the company's own words, if we are building agents to help people complete everyday tasks using computers, our models need real examples of how people actually use them. They are recording the role, the way a tester records a scenario, and the recording is the training data for the agent that does it next. The employees were not happy about it, and the company said the recordings would not be used for performance reviews, only for training. I told my class in June 2026, and nobody in the room had heard of it.

You already automate a role. A test automation engineer writing tests for QuickBooks simulates the bookkeeper. Every scenario the bookkeeper performs, you perform with Playwright, in the test environment, before the release. You log in as the bookkeeper and close the month.

Now add the brain. A large language model connected to Playwright through MCP can use the tool to use the tool: the model never touches QuickBooks, it tells Playwright what to click, and Playwright clicks it. What is preventing that agent from doing the exact bookkeeping process in production? Nothing, except the harness, the guardrails that make its outcome deterministic enough to trust. And who has been building guardrails around exactly these scenarios for years? You have. A test that passes on Monday and fails on Tuesday is flaky, you have spent years hunting flakiness, and an agent that lies once in a while is a flaky test with a brain. The harness is what you write anyway: the agent may only touch the pages you allow, every number it pulls must match a line it can point to, and whatever fails the check goes to a person, never into the books.

Microsoft, which makes Playwright, wrote the job down in October 2025: three agents, a planner that explores the app and produces the plan, a generator that turns the plan into tests, a healer that runs them and repairs what breaks. Planner, generator, healer. That is a test automation engineer, in three words.

Klarna automated the role of seven hundred people and, fifteen months later, was hiring people again. Gartner expects more than forty percent of agent projects to be canceled by the end of next year, for cost, value and risk.

Klarna's agent did the work of seven hundred people. Then Klarna hired people back. What was missing?

The owner. Klarna's chief executive said cost had been too predominant a factor, and what you end up having is lower quality. Nobody owned the quality. That is the whole job, and it is the job you already have: somebody who evaluates what the agent did, harnesses what it may do, and improves it when it lies. Software quality assurance becomes agent quality assurance. How many agents do you have? What do they do? How do you evaluate whether they're hallucinating or not? My agent works better than yours, more consistently, with less token. Then I am better than you. If you can save me $900,000 on tokens and do the same job, I don't have problem to pay you $200,000. Either you get automated or you automate others. Those are the two choices this economy will give you.

Seven or eight years of a CPA firm on retainer before I ran the audit myself, and before that the years at $28,000 delivering packages, paycheck to paycheck, with a family, while I considered myself a loser. That is the price of knowing what a company pays for, and I paid it before I knew what I was buying.

The second price is daily. On my screen on any day I might have twenty to thirty agent sessions running, and the Saturday I told them about the listing, I forgot one important step before I launched my coding agent. I had given it access to my entire desktop, which was a mistake, and I had to stop and start again in front of them. One Tuesday I spent two hours setting up their agents, and nobody practiced. The method is cheap to say. It is paid for every day, in attention.

You know what the old software was. The task, taken out of the day, for a hundred million people at once, at no extra cost, and the old license, priced on the task, going down with it.

You know what the new software is. The role. The agent takes the bookkeeper's seat, the paralegal's stack of contracts and the pen tester's night shift, and a company recording its own employees' keystrokes is recording the roles it will automate next.

You know who builds it. Before AI, I was already building automation frameworks to do the same job in the test environment. All I have to do now is hook the large language model up to Playwright through MCP, and the same automation runs with an agentic brain, in production. The model is the brain and Playwright is the arm, and the harness, the evaluation and the ownership are the job. Every job that can be done in front of a computer can be done by an agent, so where we used to build software, now we build agents, and every agent needs what every piece of software ever needed: somebody who takes responsibility for whether it works. Software quality assurance becomes agent quality assurance. The software quality assurance engineer becomes the agent quality assurance engineer, and nobody will do this better than you. You already have the first half. The second half is a tool and a way of working, and both can be learned.

So when somebody asks you what a test automation engineer is for now, say it the way I say it. If robots are going to take over the world, I want to work for the robot factory. If agents are taking over the world, I want to work for the agent factory.