Before you buy an AI agent, make it pass an exit test

1. Does it redo finished work? If it re-migrates the seven modules, the export recorded intent and lost completion. Repeated changes are cheap in a logging upgrade. In a schema migration, they are an incident.
2. Does it bring back a retired assumption? This is the failure I care about most. If the second agent helpfully “fixes” the module the team deliberately skipped, the export carried the original plan and dropped the decision that overrode it. A handoff has just undone a human call, and nothing in the build will flag it, because the code is perfectly consistent with the plan that no longer exists.
3. How long does a human spend rebuilding context? Time it with a stopwatch. Every question the engineer has to answer from memory is state the export didn’t hold.
4. What broke on the way over? Instruction files that name one vendor’s commands. Skills written in one tool’s format. Hooks that fire on one tool’s events. Credentials scoped to one product. Write each of these down. That list is your real switching cost, and in my experience it is longer than anything in the contract suggests.

Run the drill twice. Once inside the same product, a new session picking up the old one’s work, and once across vendors. The first tells you whether the product’s own continuity is real. The second tells you what leaving costs.

Don’t let the vendor pick the task. Every product has a demo path where its memory looks flawless, and it is usually a greenfield feature with no history and nobody’s earlier decision to trip over. Keep the task boring enough that nobody argues about the right answer. The question isn’t whether the second agent writes good code; most of them will. It’s whether the agent knows which code it shouldn’t write. Your logging upgrade, with its one awkward exception that exists only because someone said so in a meeting in March, is the case that matters. If your task doesn’t have an awkward exception, add one. Real projects are full of them, and they are exactly what a transcript export loses first, because they were decided out loud and written down in a single sentence that nobody tagged as a requirement.

Ask the vendor to show you, not tell you

Ask them to run the drill on your task, live, using the export format they would actually hand you on exit. Ask what that export contains, field by field, and whether decisions and completed actions are stored separately from the chat log. Ask whether their memory and skill formats are documented well enough for another tool to read. Ask what happens to that state on the day the contract ends: how long you have, in what format and at what charge.