Chapter 11 of 32

Part 2. Check yourself

Answer out loud before looking back.

One. Why is confident invention a property of the mechanism rather than a bug?

Two. A model is right ninety-nine times in a hundred. What do you need to know before you can say whether that is acceptable?

Three. Why might a model score well on a public benchmark for reasons that do not help you?

Four. What is wrong with using a model's agreement as evidence that your design is sound?

Five. An agent reports that all tests passed. What do you ask for, and why is the summary not it?