Built from real classes, checked like software
How we turned our live classes into 61 short lessons on AI agents, and how we made sure they are accurate, practical and not AI filler.
In classDay 1
- 580
“When I say peanut butter, the next word that came to your mind, number one, concrete. Number two, zebra. Number three, moon. Number four, jelly.”
The instructor, 36:27 into the class
In the lessonLesson 1.1
I’ll say two words: peanut butter. Which word comes next: concrete, zebra, moon, or jelly?
Then a two-second pause, so you answer before the lesson does.
Comes from line 580 of the class
- The quote matches the class word for word
- The lesson points back to the line it came from
- The video passed all 13 checks
The short version
Yes, AI helped make these lessons. AI wrote the first drafts from recordings of our live classes, and an AI voice reads them. That is exactly why we checked everything, the same way we teach you to check an AI agent’s work.
- 9 in 10parts of the lessons come straight from a real class
- 289problems a second reviewer found before recording
- 61 of 61videos passed all 13 automatic checks
- 3.9 vs 2.6out of 5 in a blind test against free official lessons
What you will learn
61 short lessons in seven modules. Each lesson is about 6 minutes long, and the whole course runs about 6 hours.
1. How AI and agents work
- 1.1Why LLMs hallucinate: prediction, not retrieval
- 1.2Why AI flatters you, and why you demand proof
- 1.3What makes an agent: the parts and the loop
- 1.4Provider, model, client: anatomy of an AI call
- 1.5Context window: the apprentice with a short memory
- 1.6Full context? Hand over to a fresh session
- 1.7Why 99% isn't safe: agent harnessing
- 1.8Agent-first software: tools need an agent interface
- 1.9From test automation to job automation
2. Models and cost
- 2.1Match the model and effort to the task
- 2.2Activity inventory: spend tokens where they pay
- 2.3Why AI use costs money, and three habits that save it
- 2.4Cap the bill: turn caps, subagent caps, daily limits
- 2.5When the agent goes rogue: stop and rescope
- 2.6Open-weight models and who sees your data
- 2.7Don't depend on one AI vendor
3. Agent building blocks
- 3.1MCP: the USB cable between AI and tools
- 3.2Webhook, agent, MCP: trigger, brain, tools
- 3.3Instructions vs skills: the job and the recipe
- 3.4Put the fixed steps in scripts inside a skill
- 3.5Pave the paths: which workflows deserve a skill
- 3.6Mine your session logs for skills and guardrails
- 3.7Why one agent per job works better
- 3.8Scope a subagent: one job, minimal tools, hard limits
- 3.9Orchestrators and specialist subagents: one coordinator, many specialists
- 3.10Context engineering: one clean page per agent
4. Guardrails and security
- 4.1The rules file every agent reads first
- 4.2Where your agent's rules live and apply
- 4.3Permission prompts and auto-approve: what you give up
- 4.4A question is not a go-ahead
- 4.5Test your deny list by trying to break it
- 4.6Agent keys: least privilege, one per job, expiry
- 4.7Prompt injection through your test data
- 4.8Downloaded MCP servers and skills as Trojan horses
5. Verification: gates and evidence
- 5.1Gates, not prompts: maker, checker, script, human
- 5.2Deterministic scripts check probabilistic agents
- 5.3Evidence over self-report: logs and traces
- 5.4Make agents cite sources, then verify them
- 5.5Traceability matrix: no missing, no invented tests
- 5.6From testing to evaluating agent work
- 5.7Human in the loop: approve the plan first
- 5.8Review layers: agents check, humans sign off
6. The QA workflow with agents
- 6.1The agent-era test lifecycle: plan, generate, run, heal
- 6.2The QA assembly line with agents: story to defect
- 6.3Test cases from criteria, code and documentation
- 6.4contract.md: one test-ID list for dev and test agents
- 6.5Fixtures and gates keep agent-written tests consistent
- 6.6Failed test: script bug, flaky, or real bug?
- 6.7Judge agent-written unit tests and coverage
- 6.8Migrate a legacy suite to Playwright with agents
- 6.9Stuck? Ask your agent, then judge the answer
- 6.10Root-cause a confusing failure with your agent
7. Building an app with agents
- 7.1Vibe coding vs spec- and test-driven development
- 7.2Let the AI interview you: write the spec first
- 7.3Scope your first build: domain, MVP, AI where essential
- 7.4Sessions are the new sprints
- 7.5The four-piece stack behind an agent-built app
- 7.6Wire outside services: account, key, agent
- 7.7Don't rebuild Scrum with agents
- 7.8Ship agent-written changes with a rollback path
- 7.9Bring agents to work: risk, sandbox, pilot
Where the lessons come from
Every lesson starts in a real classroom. We recorded nine live sessions, read every line of what was said, and picked out 74 topics worth teaching. The first 61 became these lessons.
- 1Record9live sessions recorded
- 2Pick74topics found in class
- 3Write61lessons written from class
- 4Check13automatic checks on every video
Across all lessons, 9 in 10 parts come straight from what was said in class. The rest are things we added to help you learn, like definitions and practice. Every quote we used was checked against the class transcript, and all 435 matched word for word. A check found no student names in the material the lessons were written from.
How we checked them
Sounding right is not the same as being right. So every lesson went through three checks.
- Fact check
We flagged 324 statements that could be wrong or out of date. We cut 173 of them and rewrote 108 so they say only what the source supports.
- A second reviewer
A separate AI reviewer read the scripts next to the class transcripts, looking for anything unclear, wrong or not true to the class. No script passed on the first try. The reviewer found 289 problems, and we fixed the scripts before recording.
- 13 automatic checks
Each finished video then had to pass 13 automatic checks. A video goes out only when it passes all of them. All 61 did.
- Short, clear sentences
- A steady speaking pace
- The voice matches the script
- Full HD picture at the right length
- The same volume in every video
- No blank screens
- No private data on screen
- Always something moving
- The voice stays above the music
- Nothing cut off at the edges
- No stray sounds between sentences
- No clicks or pops
- A final look at every scene
How every lesson teaches
Watching is not learning. So every lesson uses the same seven steps to get you thinking and doing. Here they are in lesson 1.1, which explains why AI sometimes makes things up (experts call this hallucinating). It compares an AI model to a caveman who predicts the weather by looking at the night sky.
- Guess first
Pause and predict: can hallucination be fixed the same way?
- A picture from everyday life
Imagine a caveman watching the night sky. He notices clear nights come before sunny days, and cloudy nights before rain.
- See where it breaks
Now break it. One night is cloudy, yet the next morning is bright and dry.
- Quick quiz
Quick check: three questions. Answer each question yourself before I reveal mine.
- Use it at work
At work, treat every AI answer like the caveman’s forecast: likely, but not certain. Check it before anyone relies on it.
- Try it this week
This week, ask a chatbot the same factual question in three fresh chats. Compare the answers, then check one fact against a trusted source.
- Remember this
So remember: a language model predicts; on its own, it doesn’t look things up. In one sentence: when its prediction meets the truth, it looks smart; when it misses, it hallucinates.
How they compare
We also ran a blind test. We took seven of our lessons and seven free official lessons on the same topics from major AI and testing-tool companies. We hid which was which, and two AI judges scored each lesson from 1 to 5.
- Our lessons
- Free official lessons
Show the numbers
| Measure | Our lessons | Free official lessons |
|---|---|---|
| Accurate | 4.07 | 3.21 |
| Uses real examples | 4.57 | 3.07 |
| Built around real problems | 4.00 | 2.57 |
| Shows, not just tells | 4.07 | 2.86 |
| Makes you practice | 4.93 | 1.07 |
| Easy to follow | 4.43 | 3.29 |
| Useful at work | 4.07 | 2.57 |
| Up to date | 4.64 | 3.79 |
| Overall | 3.93 | 2.64 |
Our lessons scored higher on every measure, 3.9 against 2.6 overall. The biggest gap was practice: 4.9 against 1.1.
Skills most courses skip
We looked at 47 other courses: 17 on AI agents and 30 on AI for software testing. Few of them teach the everyday skills that keep AI work safe, affordable and useful on a QA team.
- Keeping AI costs under control2 of 47 courses teach it in fullOur lessons 2.1-2.5
- Protecting your data and not depending on one AI company3 of 47 courses teach it in fullOur lessons 2.6, 2.7
- Keeping agents safe: permissions, keys and tricks hidden in data6 of 47 courses teach it in fullOur lessons 4.3-4.8
- Finding out why a test failed: a broken script, a flaky test or a real bug0 of 30 AI testing courses teach it in fullOur lesson 6.6
- Tracing every test back to a requirement0 of 30 AI testing courses teach it in fullOur lesson 5.5
- Moving an old test suite to a new tool with agents0 of 30 AI testing courses teach it in fullOur lesson 6.8
To be fair: we chose these skills from our own course outline, so we cover all of them by design. The point is how few other courses do.
What we are still working on
No course is ever finished. Here is what is not done yet:
- Captions and written transcripts for every lesson
- Graded practice labs for each module
- A scored quiz built from each lesson’s quick quiz
- A review by an outside QA expert, not just AI
- Tightening the few lines the judges said go further than the facts
- Measuring what students learn and use at work
How we measured this
- Lesson parts
- Each script is split into short parts, 2,216 in all. Each part lists the class lines it comes from or is marked as added. 92.1% cite class lines; the lowest single lesson is at 73.8%.
- Quotes
- 435 of 435 quotes matched the class transcript word for word.
- Fact check
- 324 statements flagged: 173 cut, 108 rewritten, 31 left out, 12 checked against a named source. The scripts also record checks of 714 other statements.
- Review
- 60 of 61 scripts have a review on file. Findings: 144 on clarity, 116 on accuracy, 29 on staying true to the class.
- Automatic checks
- 13 per video, including speech recognition (median match 99.9%), loudness, and text recognition that scans frames for private data (11,398 frames). We tested the script check by planting 11 mistakes on purpose, and it caught all 11.
- Blind test
- Pairs and scoring rules were written down first. Lessons were coded and shuffled, and two AI judges scored them separately on eight measures. Their scores agreed closely: a correlation of 0.885, and all 126 paired scores within one point.
- Course review
- 47 courses, read from their public pages and marked as teaching each topic in full, in part or not at all.
- Date
- Data as of November 8, 2026.