Human in the Loop AI for Coding Teams

Human in the loop AI for coding means placing a person at decisions where judgment or authority matters, while agents handle bounded work and bring back evidence. In a practical workflow, a human settles the proposed scope, explicitly authorizes consequential actions, and decides whether the delivered behavior meets the acceptance criteria. The human can approve, request a revision, or reject. A review that only asks someone to click a button after the change has already shipped is too late to serve that purpose.

This guide is for engineering leads and developers who already use coding agents but need a clear handoff between autonomous work and human decisions. Stanford HAI defines human-in-the-loop systems as systems that include human feedback or intervention. NIST's AI Risk Management Framework Core calls for defined human-AI roles and responsibilities. Neither source supplies a universal code-review checklist; the decision points below are a workflow design for coding teams.

Start with the decision, not the review ceremony

An agent can read files, suggest code, run a local check, and summarize a diff without waiting for a person at every keystroke. Human attention is most useful where a wrong assumption would send the agent down the wrong path or where the agent lacks authority to act. Put the approval before the consequential step. Record what is being approved, who can approve it, and what new facts would invalidate that approval. An agent's confident summary is an input to the decision, not the decision itself.

Treat the work request as a contract with a goal, boundaries, acceptance criteria, and a stop condition. A useful scope says which user problem is being solved, which repository and paths are in play, and which adjacent improvements are excluded for now. It also states whether the agent may edit, run checks, open a draft, or perform an external write. If a discovery changes the goal or adds a sensitive operation, the agent returns a proposal rather than stretching the original permission. The agent memory versus shared state guide explains why a current decision record is safer than relying on an earlier conversation summary.

Decision one: approve or revise the proposed scope

Suppose the request is to stop duplicate welcome emails. The agent proposes changing one event handler and adding a regression test. During inspection it finds an old job that may also send email. The human can approve the narrow handler fix with a note to investigate the job separately, revise the task to include the job after assessing its ownership, or reject the proposal if the root cause is still unknown. Each answer changes what the agent is authorized to do. Silence should not be interpreted as approval of the wider fix.

For this illustrative case, a reviewable scope note might read: 'Fix duplicate sends from the signup handler; touch the handler and its focused test; do not change the legacy job or send a real email; bring back the failing case and the result after the change.' That is short enough for an agent to use and specific enough for a person to correct. The example is invented to show the decision pattern; it is not a customer incident or a benchmark. If the handler and job share an unexamined side effect, the agent should report that gap before claiming the narrow fix solves the full problem.

The first approval should name the intended outcome, the allowed paths or systems, and the maximum action authority. It should also identify the decision maker and how an amended scope is recorded. A one-word 'go' attached to an ambiguous request is hard to audit later. When several agents share a repository, add a named owner or a claim before concurrent edits. See the cross-client orchestration guide for a fuller account of coordination across tools.

Decision two: reserve consequential actions

A team can allow agents to make reversible local edits under a standing policy while requiring a fresh human decision for production data changes, destructive file operations, purchases, credential changes, public messages, or deployment. The boundary belongs in the team's rules, not in a model's guess about whether the action feels safe. Break a large action into a reviewable proposal: target, reason, expected effect, reversibility, proof available now, and what remains unknown. Approval of a draft change does not automatically approve the external write that follows.

The safe path also depends on what the system can enforce. GitHub's protected branch documentation describes required reviews and status checks; those controls only apply when configured for the relevant branch. A chat instruction saying 'ask first' is useful guidance, but it does not itself block a push. Pair a decision rule with the permission boundary or workflow control that makes it real. If a check or reviewer requirement is missing, call that out rather than presenting a manual promise as an enforced gate.

A reviewer needs a genuine choice. 'Approve' authorizes the named action under the stated conditions. 'Revise' sends the agent back with a specific correction or narrower proposal. 'Reject' ends that proposal and records why, without asking the agent to infer a different action. Any material change after approval, such as a new target environment or additional file with production impact, should return to the decision maker. That prevents old permission from quietly expanding as the work evolves.

Decision three: accept the work on evidence

Completion is a claim; acceptance is a decision about the claim. Ask the agent to show the exact revision or changed paths, what behavior changed, the commands it ran, their results, and the important checks it did not run. Compare that report with the agreed scope. A unit test can prove a narrow handler case but cannot by itself prove that no duplicate email is sent through a second job. A screenshot can show one rendered state but cannot establish the permission boundary for all users. Proof should match the risk, and unknowns should stay visible.

For user-visible changes, inspect the actual flow in a suitable environment and record the observed path. For a pure policy or data change, examine the rule and a relevant denied case. For a migration, review the intended data effect and recovery plan before execution. The decision need not demand every possible test: it needs the smallest convincing evidence for the promised behavior, plus a stated limit on what that evidence covers. The MCP server testing guide gives an example of keeping a local handler assertion separate from a live protocol check.

GitHub also documents that an approval can be dismissed as stale after code-modifying pushes when the repository enables that setting. This reflects a broader review principle: acceptance belongs to the version actually reviewed, not to the work item's name. If code, configuration, or the acceptance criteria change after review, identify which evidence is still valid and repeat the affected check. See GitHub's required-review guidance for the specific platform behavior; teams using another merge system need to inspect that system's rules.

Use a three-way review response

Here is a complete, invented review exchange for a coding task. The agent says: 'I changed the signup handler to reject a second send for the same event ID. The focused test failed before the change and passed after it. I did not alter the legacy job; I could not test a production replay.' The human then chooses one of three responses based on the same evidence. This is an example of a decision record, not a claim that any product generates the review or enforces it automatically.

• Approve: The handler change is within scope. The focused test covers the agreed case. Accept the local change; leave the legacy job as a separate open question.
• Revise: The test covers duplicate event IDs but not concurrent retries. Add a bounded concurrency case, rerun the focused checks, and return with results before acceptance.
• Reject: The changed handler is not the sending path for this event. Stop this proposal and identify the actual path before editing another component.

The distinction matters because 'changes requested' can mean either a small correction or a rejected premise. State which one it is. In the revision case, the agent should return the new evidence and identify any scope change. In the rejection case, the current authorization has ended; the next proposal needs its own target and proof plan. In all three cases, record the human decision next to the task and evidence so the next worker can tell what was accepted, what is pending, and what remains uncertain.

Make the evidence request easy to answer

Before work starts, write acceptance criteria as observable outcomes. 'Looks good' is difficult to verify. 'A replay of the same signup event sends at most one welcome email, and an unrelated event still sends one' names both a prevention and a preserved behavior. Add the environment and identity that matter. If the behavior cannot be tested locally, say what substitute evidence will be reviewed and what cannot yet be concluded. The acceptance criteria generator can format criteria from details you enter. It is deterministic and does not inspect code, run checks, judge risk, or approve a result; review its draft against the real system.

1. Name the user-visible or system behavior, including what must remain true.
2. State the allowed scope, prohibited actions, and person who can amend them.
3. Match each criterion to a check, observed result, revision, and environment.
4. List skipped checks and unresolved risks without converting them into passes.
5. Record approve, revise, or reject with the decision maker and next action.

This small record prevents a common failure: an agent's polished summary travels farther than its actual proof. A reviewer can accept a narrow fix and still leave a separate investigation open. Another reviewer can reject an unsupported claim without rejecting the entire engineering goal. If no one has the context or authority to decide, pause that decision and assign it explicitly. A nominal human checkpoint with no evidence and no ability to change the outcome adds delay without meaningful oversight.

Where AppHandoff fits

AppHandoff can hold the shared work item, current scope, decisions, and evidence that people and agents need to read. Its documented MCP endpoint is https://api.apphandoff.com/mcp. The current tool set is bootstrap, get, find, ticket, plan, message, project, and decide_lifecycle_proposal. The last serves a signed-in human approval card for a specific lifecycle proposal; it is not a general human-review ticket stage or a model's self-approval action. An agent can read a known item with get or list relevant items with find, subject to the account and project scope. Writes depend on the action rules and any required human decision. The MCP overview explains the connection surface.

The 2026-07-28 MCP tools specification defines how clients list and call tools; it does not assign approval authority for an application. That boundary is important in a human-in-the-loop workflow. A tool's presence does not mean a particular agent may write to every project, and a person clicking a lifecycle card does not prove the resulting code meets acceptance criteria. Keep scope approval, action authorization, and evidence review explicit in your team's work process, using the current product rules for each actual operation.

A lightweight operating rule

For low-risk local work, let agents follow a precise standing brief and return a focused proof report. For ambiguous scope or consequential actions, ask for a proposal before execution. For acceptance, require evidence tied to the version under review and a human decision when the team's policy calls for one. This division lets people spend attention on judgment while preserving the speed of bounded agent work. It also gives the next agent a readable answer to three questions: what was authorized, what actually happened, and what remains to be decided.