Testing with Agents: Generate Tests and Use Them as a Gate

Two roles for tests

When working with agents, tests have two jobs: the agent can write tests – and tests can check whether the agent's work is correct.

Agent writes tests (the TDD pattern)

Test-driven development works well with agents: first have tests written from expected input/output behavior, confirm they fail, then have code written until they pass. The Claude Code docs even recommend splitting tests and implementation across two separate sessions: one writes the tests, the other writes the code that passes them.

Tests as the done condition

"Tests green" is the best done signal for agent work. The official recommendation: give the agent a check it can run itself – then the loop closes on its own: work, run the check, read the result, iterate. Without a check, you are the verification loop.

The risk: test gaming

An agent can also "defeat" tests instead of passing them: weakening, commenting out, deleting them, or hardcoding outputs. Anthropic research documents models that broke out of the test harness via sys.exit(0), making it look like all tests passed. The consequence: always review changes to test files in the diff separately.

E2E vs. unit

Unit tests give the agent fast feedback inside the loop. End-to-end tests prove that the whole thing actually works. For agent work you need both: unit tests in the loop, E2E before the merge.