Reliable multi-step AI agents need more than a capable model. They need explicit state: what the agent knows, what it plans to do, which actions have completed, and how to recover when a tool fails. This guide provides a reusable checklist for designing memory, plans, checkpoints, persistence, and human approval into an agent workflow.
Overview
AI agent state management is the practice of recording and restoring the information an agent needs to continue a task safely. In a simple chat interaction, the current conversation may be enough. In a multi-step workflow, however, the agent may need to remember a user preference, preserve a task plan, store a tool result, avoid repeating a completed action, or pause until a person approves a consequential step.
A useful design separates four kinds of state:
| State type | Purpose | Typical lifetime | Good fit |
|---|---|---|---|
| Short-term context | Recent messages, tool results, and instructions needed for the current decision | One turn or run | Conversation continuity and immediate tool use |
| Persistent memory | Stable facts or preferences that may help in future sessions | Across sessions | User preferences, approved facts, and recurring settings |
| Task plan | Ordered or conditional steps required to achieve a goal | Until the task ends | Research, data processing, and workflow automation |
| External state store | Durable records, locks, events, artifacts, and audit information | Defined by retention needs | Resuming jobs, coordinating workers, and operational recovery |
Do not treat the model's context window as a database. Context helps the model reason about supplied information, but durable state should live in an appropriate store that your application can validate, query, version, and secure. Structured output and tool schemas can make the boundary clearer; see this guide to structured output prompts for related patterns.
A practical state model
Start with a state object that distinguishes identity, goal, progress, evidence, and control information. For example:
{
"run_id": "run_123",
"user_id": "user_456",
"status": "waiting_for_approval",
"goal": "Prepare a customer renewal summary",
"plan": [
{"id": "collect_data", "status": "completed"},
{"id": "draft_summary", "status": "completed"},
{"id": "send_email", "status": "blocked"}
],
"artifacts": ["summary_v2"],
"last_checkpoint": "cp_008",
"pending_action": {
"type": "send_email",
"idempotency_key": "renewal_user456_v2"
},
"errors": [],
"schema_version": 1
}The exact fields will vary, but the principles are consistent: use stable identifiers, make status explicit, record the last safe point, and preserve enough information to explain what happened.
Checklist by scenario
For a single multi-step run
- Define the goal and completion criteria before the first tool call.
- Represent the plan as separate steps with identifiers and statuses such as
pending,running,completed,failed, orblocked. - Store tool inputs, outputs, timestamps, and relevant error details outside the prompt.
- Checkpoint after meaningful progress rather than only at the end.
- Pass the model a compact state summary instead of replaying every historical event.
- Set limits for steps, retries, tokens, time, and tool calls.
For an interrupted or resumable run
- Assign every run a durable
run_idand every action anaction_id. - Persist the state before acknowledging completion to the caller.
- Record whether an action was planned, started, completed, or has an unknown outcome.
- On restart, load the last checkpoint and determine whether the next action is safe to retry.
- Use idempotency keys for operations that could create duplicates, such as sending a message or creating a record.
- Route ambiguous outcomes to a reconciliation step or human review instead of guessing.
For persistent memory across sessions
- Store only information with a clear future benefit.
- Attach provenance, such as when the fact was recorded and how it was obtained.
- Separate user-provided facts from model inferences.
- Give users or operators a way to correct, remove, or ignore a memory.
- Apply a relevance and freshness check before injecting memory into a prompt.
- Keep sensitive information out of general-purpose memory unless the application has a specific, controlled reason to retain it.
For human approval
Approval should be represented as state, not as an informal instruction in the conversation. A reusable approval record can look like this:
{
"approval_id": "approval_789",
"run_id": "run_123",
"action": "send_email",
"proposal": {"recipient": "customer@example.com", "subject": "Renewal summary"},
"risk": "external_communication",
"status": "pending",
"requested_at": "2026-01-15T10:00:00Z",
"expires_at": null,
"decided_by": null,
"decision_reason": null
}After approval, revalidate the proposed action against current state. An approval for an old draft should not automatically authorize a changed recipient, amount, query, or document.
For tool-heavy workflows
Keep tool execution separate from model reasoning. The model can propose an action, but application code should validate its schema, permissions, dependencies, and current version of state before execution. For function-calling design and error handling, review this guide to LLM tool use. If the workflow calls external services or webhooks, the recovery patterns in this automation guide are also relevant.
What to double-check
Checkpoint boundaries
A checkpoint should represent a state from which the application can safely continue. Save it after a completed, durable action and before starting an action that may have side effects. A checkpoint record commonly includes the state version, completed step IDs, artifact references, pending action, and a hash or revision number to detect stale writes.
Concurrency and stale state
Two workers can load the same state and make incompatible changes. Use optimistic concurrency, a version field, or a controlled lock. Before writing, verify that the version you read is still current. If it is not, reload and reconcile rather than silently overwriting the newer state.
Retry behavior
Classify failures before retrying. A temporary network error may be retryable; invalid arguments, permission failures, and policy blocks usually require correction. Add bounded exponential backoff where appropriate, but do not retry an external side effect unless the operation is idempotent or its outcome can be checked.
Context assembly
Build the prompt from deliberate components: system rules, current goal, relevant memory, current plan, recent events, tool constraints, and the required output schema. Exclude stale or unrelated history. For long conversations, streaming and cancellation behavior also affects state transitions; see this article on streaming LLM responses.
Evaluation and observability
Log state transitions, not just final answers. Test interrupted runs, duplicate delivery, malformed tool output, conflicting instructions, expired approvals, and missing records. A regression suite with golden tasks can compare both the final result and the path taken; the LLM evaluation test suite guide offers a useful testing framework.
Common mistakes
- Putting everything in the prompt: Large transcripts are expensive to manage and still do not provide durable, transactional storage.
- Using memory as truth: Stored facts can become outdated or can reflect an incorrect model inference. Track provenance and validate important facts.
- Saving only the final answer: Without intermediate state, a failed run may need to restart every tool call and may repeat side effects.
- Retrying blindly: A retry can duplicate an email, payment, ticket, or database update. Use idempotency and outcome checks.
- Allowing the model to change control fields: Status, permissions, approval decisions, and retry counts should be controlled by application code.
- Ignoring conflicting instructions: Separate trusted application instructions, user requests, retrieved content, and tool output. Test for prompt injection, especially when external text enters the context. Use the prompt injection prevention checklist.
- Failing to version schemas: A workflow change can make old checkpoints unreadable. Include a schema version and define migration or abandonment behavior.
When to revisit
Review AI agent state management before seasonal planning cycles, after adding a tool, and whenever a workflow changes. Also revisit it when changing models or context-handling conventions, migrating agent frameworks, modifying memory behavior, introducing parallel workers, or adding a human approval step.
Use this maintenance checklist:
- Run interrupted-task tests from every important checkpoint.
- Verify that old state records can be migrated or safely marked obsolete.
- Test duplicate tool requests and unknown external outcomes.
- Review which memories are retained, injected, corrected, and deleted.
- Confirm that approval records expire or are revalidated when underlying data changes.
- Compare state-transition traces against expected workflows, not only final responses.
- Update prompt schemas, tool contracts, and recovery rules together.
The strongest agent architecture is not the one that never fails. It is the one that makes progress visible, limits unsafe repetition, resumes from a known point, and gives operators a clear path when automation cannot decide. Treat state as a first-class part of your AI development workflow, and multi-step agents become easier to test, maintain, and improve.