Most advice about working with AI assistants is written by people who used one for a weekend. This is the other kind: notes from twelve months of running coding agents on production work, kept in a plain text file and cleaned up here.
The context problem is the whole problem
The single biggest predictor of a good result is not the model, the prompt template or the temperature setting. It is whether the assistant has the material. A question that only makes sense if you already know the codebase, the client and the last four decisions will get a plausible, confident, wrong answer. Paste the file. Link the document. Point at the directory. A model with the material beats a smarter model without it, every time.
Scope before you delegate
Unscoped work does not fail loudly. It produces something complete-looking and subtly wrong, and it produces it fast. Before anything runs I write down three things: which files are in play and which are not, what must still be true afterwards, and the exact command that says whether it worked. Two minutes of writing saves the afternoon spent unwinding a beautifully formatted refactor nobody asked for.
Ask for the plan, then the patch
Three bullets describing the intended approach costs almost nothing and catches the misunderstanding while it is still free. In my own logs this intercepts roughly a third of tasks, and in half of those the problem turns out to be my request rather than the model's reading of it. That is information worth having before a hundred lines exist.
Read the diff, never the summary
An agent's account of its own work is the least reliable artifact it produces. Not because it lies, but because a summary is a compression written by the same process that produced the work: if it misunderstood the task, the summary describes the misunderstanding fluently. The diff is evidence. Scan the file list first — an unexpected filename is always a question.
Generated tests need a test
Tests written after reading the implementation tend to assert that the code does what the code does. When I read a generated test I ask one question of it: what change to the source would make this fail? If I cannot answer, the test is decoration, and decoration is worse than nothing because it looks like coverage.
Someone still has to stop
Left running, an agent will refactor, generalize, add a configuration layer nobody requested and then write tests for the layer. There is no internal sense of enough. Deciding that the work is finished is now one of the more valuable things a person does on a software team, and it deserves to be a deliberate act rather than a side effect of exhaustion.
The pattern behind all of it
Everything that genuinely moved off my plate this year shares a shape: bounded input, checkable output, and being wrong is cheap and visible. Everything that stayed fails at least one of those. That three-part test has predicted "will this work" better than any benchmark I have read, and it takes ten seconds to apply.
Longer walkthroughs of these workflows — agents, Model Context Protocol servers, and getting real output from neural networks on ordinary tasks — live at arsentev.ai.
Sign in to leave a comment.