First run the tests

⋅

Agentic Engineering Patterns >

Automated tests are no longer optional when working with coding agents.

The old excuses for not writing them – that they’re time consuming and expensive to constantly rewrite while a codebase is rapidly evolving – no longer hold when an agent can knock them into shape in just a few minutes.

They’re also vital for ensuring AI-generated code does what it claims to do. If the code has never been executed it’s pure luck if it actually works when deployed to production.

Tests are also a great tool to help get an agent up to speed with an existing codebase. Watch what happens when you ask Claude Code or similar about an existing feature – the chances are high that they’ll find and read the relevant tests.

Agents are already biased towards testing, but the presence of an existing test suite will almost certainly push the agent into testing new changes that it makes.

Any time I start a new session with an agent against an existing project I’ll start by prompting a variant of the following:

First run the tests

For my Python projects I have pyproject.toml set up such that I can prompt this instead:

Run "uv run pytest"

These four word prompts serve several purposes:

  1. It tells the agent that there is a test suite and forces it to figure out how to run the tests. This makes it almost certain that the agent will run the tests in the future to ensure it didn’t break anything.
  2. Most test harnesses will give the agent a rough indication of how many tests they are. This can act as a proxy for how large and complex the project is, and also hints that the agent should search the tests themselves if they want to learn more.
  3. It puts the agent in a testing mindset. Having run the tests it’s natural for it to then expand them with its own tests later on.

Similar to “Use red/green TDD”, “First run the tests” provides a four word prompt that encompasses a substantial amount of software engineering discipline that’s already baked into the models.

Tags: tdd, testing, llms, ai, generative-ai, ai-assisted-programming, coding-agents, agentic-engineering

Liked Liked