Yeah. The method above does work quite well in practice. I would not say that I currently have a complete, 100% explanation of why it works, but neither are we starting from zero. The following is, well, sort of a manual explaining the benefits of using it, written by me and ChatGPT:
The most useful way to understand this is not as a magic prompt that makes the underlying language model suddenly reliable.
It changes the whole working system around the model.
In this setup, the relevant unit is not just the LLM. It is a human-mediated coding agent consisting of:
- the model doing language and reasoning work,
- the instructions and current chat state,
- the bounded packet it proposes,
- the terminal and repository it interacts with indirectly,
- the human who retains execution authority,
- and the tests, diffs, logs, and acceptance criteria used to decide whether to continue.
This matches the broader use of agent in current tooling: for example, the OpenAI Agents SDK describes agents in terms of models plus instructions, tools, guardrails, sessions, tracing, and human approval, while Anthropic’s agent-evaluation guide explicitly treats the model and the agent harness as a joint evaluation unit.
So the short explanation is:
Mechanical coding turns a broad generation request into a bounded loop of proposal, authorization, execution, observation, verification, and either continuation or stopping.
That produces three different benefits:
- Grounding: unsupported assumptions are replaced sooner by observations from the actual environment.
- Correction: failures become evidence that can change the next decision.
- Containment: if the agent is wrong, the permitted scope and resulting damage are smaller.
It does not make the model infallible, and it does not make every terminal result a proof of full correctness. It makes errors easier to expose, localize, interrupt, and recover from.
A compact way to state the division of responsibility is:
The terminal answers, “What actually happened in this execution?”
The acceptance criteria answer, “Was that the right thing to happen?”
A practical default route
For future readers, I would use approximately this sequence:
- Define the intended outcome and the evidence that would count as success.
- Start with a read-only audit and, where applicable, reproduce the reported problem.
- Allow “no change is required” to be a valid result of the audit.
- Propose one bounded packet with an explicit purpose and scope.
- Review the actual commands and affected files before execution.
- Run it in the narrowest practical environment and return the complete output, exit status, and resulting diff.
- Classify a failure before editing again: code, environment, permission, harness, or verifier.
- Verify with at least one check that is not freely rewriteable by the same packet.
- Retry only when there is new evidence, a new hypothesis, or a materially different experiment.
- At intervals, check the project globally so that locally passing packets do not accumulate into architectural drift.
A useful packet can be treated as a lightweight contract:
Purpose
Observed preconditions
Allowed files and operations
Forbidden changes and side effects
Exact commands
Expected observations
Pass criteria
Failure and stop conditions
Rollback or compensation plan
Remaining uncertainty
The packet is therefore not merely “a small amount of code.” It is a short, observable, reviewable, and preferably reversible experiment.
| Component |
Primary role |
| ChatGPT / coding agent |
Maintains the working instructions and proposes the next action |
| Model |
Performs the language and reasoning work inside the agent |
| Packet |
Defines the bounded unit of proposed work |
| Human |
Holds authorization, semantic judgment, and stop control |
| Terminal |
Executes commands and returns observations from the real environment |
| Tests, checks, and diffs |
Provide partial evidence about the result |
| Acceptance criteria |
Connect operational evidence to the user’s actual requirement |
| Sandbox and permissions |
Enforce what the agent can reach even if judgment fails |
| Progress record |
Preserves verified state across long sessions or handoffs |
1. Why this is better understood at the agent level
A bare language model maps an input sequence to an output sequence. Mechanical coding adds a surrounding control structure:
instructions
↓
proposal
↓
authorization
↓
execution
↓
observation
↓
verification
↓
continue / revise / stop
The behavior we observe therefore belongs to the combination of model, instructions, tools, environment, state management, verification, and human intervention.
This distinction matters because the same underlying model can behave very differently under different interfaces and scaffolds.
The SWE-agent paper calls this an Agent–Computer Interface. Its central point is that the commands, feedback formats, repository-navigation tools, and editing interfaces presented to an agent materially affect its behavior and performance.
This also explains why a simpler workflow may outperform a more “autonomous” one. Agentless, for example, organizes software repair into relatively explicit localization, repair, and validation stages rather than relying on an unrestricted general-purpose agent loop.
The relevant question is therefore not only:
How capable is the model?
It is also:
What can the agent observe, what actions can it select, what state persists, what evidence does it receive, and what conditions allow it to continue?
Mechanical coding changes all of those conditions without changing the model’s weights.
2. Why the read-only audit is more than preliminary housekeeping
The audit serves several independent purposes.
First, it replaces guessed project state with observed project state:
- repository location,
- language and framework,
- package and dependency files,
- available test commands,
- existing documentation,
- likely entry points,
- current failures,
- and obvious operational risks.
Second, it separates problem discovery from solution generation. Without this separation, an agent may begin constructing a solution for a repository, framework, file layout, or failure mode that exists only in its inferred picture of the project.
Third, it creates a valid abstention branch:
evidence indicates a change is needed
→ propose a bounded change
evidence does not establish that a change is needed
→ report the evidence and stop
This matters because a coding request implicitly encourages action. “Produce code” is often treated as the expected conversational outcome even when the correct engineering outcome is:
- the behavior is already correct,
- the failure cannot be reproduced,
- the problem is environmental rather than in the source,
- the requested behavior conflicts with the existing contract,
- or more evidence is required before changing anything.
The audit should therefore not be judged by whether it produces a patch. It should be judged by whether it reduces uncertainty enough to choose between acting, retrieving more evidence, escalating, or stopping.
A small qualification: failure to reproduce a problem is not automatically proof that no problem exists. It may indicate a platform difference, version mismatch, missing fixture, intermittent condition, or incomplete reproduction procedure. The result should remain:
“Not reproduced under these recorded conditions,”
rather than:
“The problem does not exist.”
3. Why returning terminal output is different from asking the model to think again
There is an important difference between:
“Review your answer and try again.”
and:
“Here is new evidence from executing your proposed command in the actual environment.”
The first asks for intrinsic self-correction. The second changes the information available to the next inference.
Research on intrinsic self-correction is mixed and task-dependent. In Large Language Models Cannot Self-Correct Reasoning Yet, models often failed to improve reasoning without external feedback and sometimes degraded correct answers. A related result, LLMs cannot find reasoning errors, but can correct them given the error location, suggests that locating the error may be a different and harder problem than revising it once useful diagnostic information is available.
Terminal execution can provide exactly that missing diagnostic information:
- a compiler identifies a file and location,
- a test identifies a violated expectation,
- a traceback identifies a failing call path,
- a type checker identifies an incompatible interface,
- a diff shows what actually changed,
- an exit status distinguishes command-level success from failure.
This is closely related to the reasoning–action–observation structure described by ReAct: actions connect the agent to an external environment, and observations can update its plan.
The useful causal story is therefore not:
The model becomes wiser because it reflected.
It is:
The next decision is conditioned on evidence that did not exist at the previous step.
That said, feedback is only useful when it is relevant and sufficiently complete. A truncated traceback, omitted exit code, wrong working directory, stale test run, or copied summary rather than raw output may support the wrong diagnosis.
A good evidence return normally includes:
working directory
relevant tool and dependency versions
exact command
complete stdout
complete stderr
exit status
files changed
diff or patch
checks run
checks not run
environmental anomalies
Repeated generation without new evidence is not a meaningful closed loop. A reasonable stopping rule is:
Do not retry the same theory under the same conditions merely by asking for another answer.
Retry when at least one of the following changes:
- a new observation becomes available,
- the suspected cause changes,
- the experiment becomes more discriminating,
- the environment is corrected,
- or a different verification channel is introduced.
4. What a bounded packet actually bounds
“Small” can be misleading if interpreted only as a line count.
A five-line change can rotate credentials, delete remote data, alter an authentication rule, or break a public interface. A larger mechanical refactor may be low-risk if it is generated, checked, and reverted deterministically.
The more useful dimensions are:
| Boundary |
Question |
| Epistemic |
How many unverified assumptions does this packet depend on? |
| Semantic |
How many behaviors is it intended to change? |
| Spatial |
Which files, modules, services, or resources can it affect? |
| Operational |
Which commands and tools may be used? |
| Temporal |
How far can work proceed before another observation is required? |
| Authorization |
What side effects are actually permitted? |
| Verification |
Can the result be checked independently and promptly? |
| Recovery |
Can the change be reverted or compensated? |
The goal is a short verification distance: the distance between making a change and receiving evidence capable of falsifying the hypothesis behind it.
This connects with the general software-engineering principle of working in small batches. DORA’s guidance emphasizes that small batches shorten feedback cycles, allow hypotheses to be tested quickly, and make course correction easier. It also notes that AI-generated large changes can be especially difficult to review and integrate safely.
A useful packet is therefore not merely small. It should be:
- independently understandable,
- directed at one semantic purpose,
- testable without completing several future packets,
- easy to review,
- capable of producing evidence that changes the next decision,
- and recoverable if the hypothesis is wrong.
There is also an opposite failure mode: packets can become too small.
Excessive fragmentation may:
- hide the overall architecture,
- encourage repeated local workarounds,
- create duplicated abstractions,
- increase approval fatigue,
- and allow every packet to pass while the project as a whole becomes less coherent.
That is why packet-level checks should be supplemented by periodic project-level checks.
5. The terminal is a source of truth—but only for the proposition it actually observed
I would interpret “the terminal proves” narrowly and operationally.
For example:
| Observation |
What it supports |
What it does not automatically prove |
pwd |
The command ran in a particular directory |
The correct repository or worktree was selected |
| File listing |
Those paths existed at that time |
The repository was understood completely |
Exit code 0 |
The process reported success |
The requested behavior is semantically correct |
| Compiler success |
The checked code compiled |
Runtime behavior is correct |
| Unit-test success |
Covered examples behaved as expected |
Uncovered cases and integrations are correct |
| Diff |
These textual changes occurred |
The changes are appropriate or complete |
| Browser check |
A visible path worked under those conditions |
Accessibility, security, portability, or all user flows are correct |
| Deployment success |
The deployment operation completed |
The release is safe or satisfies the product requirement |
The terminal is strong evidence about what happened.
The user’s requirement, specification, or acceptance criteria remain the authority for what ought to happen.
This distinction is an instance of the longstanding test oracle problem: running software is often easy; determining whether its behavior is correct can be much harder.
A test suite is a partial executable interpretation of the requirement, not the requirement itself.
This is not a merely theoretical caveat. OpenAI’s 2026 audit, Why SWE-bench Verified no longer measures frontier coding capabilities, found that many audited tasks contained material problems in their tests or problem descriptions. Tests could be too narrow, rejecting valid implementations, or too broad, requiring behavior not stated in the task.
For a practical workflow, verification is stronger when it combines different forms of evidence:
- existing regression tests,
- a fail-to-pass reproduction,
- static analysis,
- type checking,
- an end-to-end path,
- a property or invariant,
- a diff review,
- a check outside the packet’s writable scope,
- and a human-readable semantic acceptance condition.
One useful rule is:
The agent should not have unrestricted ability to rewrite the only evidence used to declare itself successful.
An agent-generated test can still be valuable, especially for clarifying an intended invariant. It is simply less independent than a locked regression test, a hidden check, or a separately defined acceptance procedure.
6. Human approval and enforced containment solve different problems
In this workflow, the human is doing real work:
- interpreting the goal,
- deciding whether a proposed packet is appropriate,
- authorizing execution,
- detecting suspicious scope expansion,
- interpreting ambiguous outcomes,
- and stopping or redirecting the process.
That means successful performance belongs to the joint human–agent workflow, not solely to the model.
However, human approval is not a perfect safety boundary.
Anthropic reports in How we contain Claude across products that users approved roughly 93% of permission prompts and became less attentive as prompts accumulated. This is one example of approval fatigue: a control may become less effective when it demands frequent low-value decisions.
For this reason, it is useful to distinguish:
- human approval, which provides judgment and intent alignment;
- hard containment, which enforces what the agent can physically reach.
Examples of hard containment include:
- a disposable container or virtual machine,
- filesystem write boundaries,
- no credentials inside the execution environment,
- network denial by default,
- read-only mounts,
- scoped tokens,
- resource and retry limits,
- and separation between planning and irreversible execution.
The OWASP AI Agent Security Cheat Sheet similarly recommends least privilege, validation of external inputs, human approval for high-risk actions, context isolation, tool limits, structured logging, and separating decision-making from execution for irreversible operations.
The practical principle is:
Use human attention for semantic and high-risk decisions; use enforceable boundaries for capabilities that should never depend on perfect attention.
The review should also be attached to the actual proposed action, not merely to the agent’s prose summary.
Before executing, inspect:
- the exact command,
- the current directory,
- the writable scope,
- network and credential access,
- whether the action is reversible,
- and what observation it is expected to produce.
Terminal and repository output can also be untrusted content
Another subtle point is that terminal output can be reliable evidence about execution while still containing untrusted text.
Repositories, documentation, build logs, issue text, downloaded files, tool responses, and dependency output can contain instructions addressed to an AI system. Those instructions should not silently redefine:
- the user’s goal,
- the packet’s permissions,
- the success criteria,
- or the authorization boundary.
A useful trust rule is:
External text may update facts about the environment, but it does not grant new authority.
7. Diagnose the layer of failure before changing code
A failed command is evidence, but it does not always mean the source code is wrong.
A practical failure taxonomy is:
| Failure layer |
Typical signs |
Appropriate next action |
| Specification |
Multiple reasonable interpretations; unclear target |
Clarify or define an acceptance branch |
| Observation |
Missing files, truncated logs, stale output |
Collect the missing evidence |
| Localization / reasoning |
Evidence is valid but the proposed cause does not follow |
Design a more discriminating experiment |
| Code / task |
Reproducible failure in the intended environment |
Propose a bounded repair |
| Environment / dependency |
Version, platform, path, fixture, or service mismatch |
Repair or record the environment |
| Permission / containment |
Operation blocked or scope insufficient |
Escalate explicitly; do not route around it silently |
| Harness / state |
Wrong working directory, stale state, context loss, tool misuse |
Repair the workflow state |
| Verification / oracle |
Check is inconsistent with the requirement or environment |
Repair or supplement the verifier |
| Human review |
Approval occurred without adequate inspection |
Increase selective friction at the risk boundary |
| Trajectory |
Each local patch passes but project quality deteriorates |
Pause for a global review |
This avoids a common anti-pattern:
command failed
→ assume code defect
→ edit code
→ create a new failure
Sometimes the correct next packet is not a patch. It may be:
- a version check,
- a clean-environment reproduction,
- a minimal failing example,
- a dependency inspection,
- a permission report,
- a verifier audit,
- or a stop-and-escalate packet.
This is where the “mechanical” aspect becomes valuable: it discourages changing several layers simultaneously and then guessing which change mattered.
8. Long-running work needs durable state outside the conversational working set
Returning output to the same chat is a good short-run mechanism because it preserves the immediate action–observation sequence.
It is not, by itself, a complete long-term state system.
Long projects encounter:
- context limits and compaction,
- requirements added across many turns,
- stale assumptions,
- contradictory decisions,
- partial implementation at session boundaries,
- premature declarations of completion,
- and handoff to a fresh model or person.
Anthropic’s Effective harnesses for long-running agents reports similar failure patterns: agents attempted too much at once, left half-implemented states, had to guess what happened in previous sessions, or later declared work complete prematurely. Their mitigation used incremental work together with progress artifacts, feature lists, clean session boundaries, and version history.
A useful division is:
chat context
= current reasoning working set
repository state
= current executable project state
progress record
= durable operational state and verified history
acceptance contract
= durable statement of what remains to be proven
For longer tasks, a compact progress record might include:
current goal
active non-goals
verified observations
decisions and reasons
files changed
checks passed
checks not run
known failures
rejected hypotheses
unresolved questions
next safe packet
A raw log is helpful, but provenance is better than an undifferentiated transcript.
The W3C PROV model provides useful general vocabulary for distinguishing entities, activities, agents, usage, generation, and derivation. A lightweight coding record does not need to implement the full standard, but it benefits from similar distinctions:
- observed,
- inferred,
- proposed,
- executed,
- verified,
- contradicted,
- superseded,
- and unresolved.
Otherwise, an old model inference may later be retrieved as though it were an observed fact.
Periodic global checkpoints should also ask questions that local tests may miss:
- Is the architecture becoming more complicated?
- Are similar workarounds being duplicated?
- Have public interfaces changed unintentionally?
- Have dependencies or permissions expanded?
- Are the original non-goals still being respected?
- Is the next change easier or harder because of the accumulated design?
- Which requirements still lack independent verification?
9. A small reproducible sanity check
Anyone interested in testing the workflow can compare it without treating a single anecdote as decisive.
A basic comparison would keep the following as constant as possible:
- repository snapshot,
- task,
- ChatGPT or agent configuration,
- available tools,
- time or token budget,
- and final acceptance procedure.
Then compare staged conditions:
| Condition |
Added control |
| A |
Broad one-shot coding request |
| B |
Explicit outcome, non-goals, and writable scope |
| C |
Read-only audit and reproduction step |
| D |
No-change / abstention branch |
| E |
Complete external execution feedback |
| F |
Bounded packet contract |
| G |
Locked or independent verification |
| H |
Periodic global checkpoint |
Useful observations include:
- unnecessary edits,
- edits outside the intended scope,
- claims of success unsupported by execution,
- number and type of failed commands,
- human interventions,
- recovery time,
- regression count,
- unresolved assumptions,
- and final semantic acceptance.
Because agent behavior varies between runs, repeat each condition rather than relying on one trajectory. Anthropic’s agent eval guide distinguishes tasks, repeated trials, graders, transcripts, outcomes, and harnesses for exactly this reason.
It is also useful to separate the trace from the outcome:
- “The agent said the task passed” is a trace observation.
- “The required state exists and independent checks pass” is an outcome observation.
This comparison would not reveal a complete internal mechanism, but it could help estimate which workflow components contribute most for a particular class of project.
10. What is well supported, and what remains provisional
Relatively well-supported functional claims
The following claims are supported by several independent research and engineering traditions:
- Agent behavior depends on the interface and harness, not only on the underlying model.
- External observations can provide correction signals that unsupported introspection lacks.
- Smaller, independently testable batches shorten feedback and recovery cycles.
- Explicit permissions and hard capability boundaries reduce the blast radius of mistakes.
- Tests and other graders are necessary but incomplete proxies for intended behavior.
- Multi-step agent performance requires evaluation of trajectories and outcomes, not only final prose.
- Long-running work benefits from durable state and explicit handoff artifacts.
Claims that should remain provisional
We do not yet have a complete account of:
- the exact internal mechanism by which each instruction changes ChatGPT’s behavior,
- the causal contribution of each line in the template,
- the optimal packet size for arbitrary projects,
- how much improvement comes from the human, the terminal feedback, the scope restriction, or the tests individually,
- how well one result generalizes across models, agent products, repositories, and task horizons,
- or whether future agent architectures will require the same control pattern.
There are also boundary conditions.
Mechanical coding can still fail when:
- the requirement is wrong or incomplete,
- the audit observes the wrong environment,
- the output is truncated or misleading,
- the verifier encodes the wrong behavior,
- the human approves without understanding,
- the permitted scope is still too broad,
- an irreversible external action is treated as rollbackable,
- or many locally successful packets accumulate into a globally poor design.
So I would describe the current explanation as functional and conditional, not complete and mechanistic:
The workflow works well because it restructures coding into short, observable, revisable experiments; limits how far unsupported assumptions can propagate; and places evidence, authorization, verification, and stopping points between successive actions.
That is already a substantial explanation, even though it is not the final explanation.
Concepts and references for further reading
These are useful search terms and starting points for readers who want to connect the workflow to existing literature: