How to Implement LLM Function Calling and Tool Use: Schemas, Errors, and Safety

Overview

Function calling, also called tool use, is an interaction pattern between an LLM application and external functions. Your application describes available tools to the model. The model may then return a structured request containing a tool name and arguments. Your server validates that request, runs the corresponding function, and sends the result back to the model. The model can use that result to answer the user or request another approved tool.

The model should not be treated as the execution environment. It proposes an action; your application decides whether that action is valid, permitted, and appropriate. This separation is the foundation of reliable LLM API integration.

A typical flow looks like this:

1. Receive the user’s request and establish the user, session, and permission context.
2. Send the conversation plus a small set of tool definitions to the model.
3. Inspect the response for text, tool calls, or both, according to the selected provider’s API format.
4. Validate the tool name and every argument against a server-side schema.
5. Check authorization, risk, rate limits, and confirmation requirements.
6. Execute the function with timeouts and bounded retries.
7. Return a concise, structured tool result to the model.
8. Repeat only while the workflow remains within a defined step and cost budget.

Provider syntax changes over time, so use the current documentation for the model and SDK you have selected. The design principles below apply across implementations.

For related workflow design, see How to Design LLM Chatbot Conversation Flows. If your application also retrieves private documents, pair tool use with a carefully tested RAG application workflow.

Checklist by scenario

1. Read-only tools

Start with low-risk operations such as product lookup, weather retrieval, internal search, or account-status queries. Define narrow inputs and return only the fields the model needs.

• Give the tool a specific name, such as get_order_status, rather than use_system.
• Describe when the tool should and should not be used.
• Mark required fields and constrain formats, enumerations, lengths, and numeric ranges.
• Filter the result server-side so secrets, unnecessary personal data, and internal errors are excluded.
• Set a timeout appropriate to the dependency and return a controlled failure when it is exceeded.

2. State-changing tools

Actions such as sending messages, modifying records, placing orders, or deleting data require stronger controls. The model may identify the intended action, but your application should remain the final authority.

• Use an explicit permission check based on the authenticated user and resource.
• Require a confirmation step for consequential or irreversible actions.
• Prefer idempotency keys so a retry cannot create duplicate side effects.
• Log the requested action, validated arguments, decision, result, and actor without recording unnecessary sensitive data.
• Offer a dry-run or preview mode where practical.

3. Multi-step workflows

A model may need to call several tools—for example, find an account, inspect eligibility, and prepare an update. Make the orchestration loop explicit rather than allowing unlimited recursion.

• Set a maximum number of tool rounds.
• Track which tools have already run and why.
• Pass only relevant outputs into the next step.
• Stop if the model repeats a failed call, changes the target unexpectedly, or requests a tool outside the allowed plan.
• Return a useful status to the user when the workflow stops instead of silently continuing.

Reusable tool-definition template

{
  "name": "lookup_order",
  "description": "Retrieve the current status of an order the signed-in user can access.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "The exact order identifier"
      }
    },
    "required": ["order_id"],
    "additionalProperties": false
  }
}

This is a conceptual template, not provider-specific syntax. Some APIs place the schema inside a separate function object or use a different field name. Keep the contract strict even when the surrounding request format differs.

Request-and-response pattern

A model response might conceptually contain a call like this:

{
  "tool_call": {
    "name": "lookup_order",
    "arguments": {"order_id": "A-1042"}
  }
}

Your application should validate the name and arguments, call the function, and send back a bounded result:

{
  "tool_result": {
    "name": "lookup_order",
    "status": "ok",
    "data": {
      "order_id": "A-1042",
      "status": "shipped",
      "estimated_delivery": "2026-11-08"
    }
  }
}

Do not assume that valid JSON means valid business input. Schema validation is necessary, but authorization and domain validation are separate checks.

What to double-check

Schema and argument validation

Parse arguments defensively. Reject unknown properties where possible, normalize formats deliberately, and validate identifiers against the current user’s accessible resources. Never construct a database query, shell command, URL, or authorization decision directly from untrusted model output.

Error handling

Use error categories that help both the application and the model respond appropriately:

• Invalid arguments: return a structured correction request or ask the user for missing information.
• Unauthorized: stop the call and do not reveal whether a protected resource exists.
• Not found: provide a neutral result and avoid repeated retries.
• Dependency failure: retry only when the operation is safe and the failure may be temporary.
• Timeout: stop the function and tell the model the result is unavailable.
• Business-rule rejection: explain the allowed next step without exposing internal implementation details.

Keep internal stack traces and credentials out of tool results. Give the model a concise, machine-readable status and a user-safe message.

Safety boundaries

• Expose the smallest useful tool set for each request instead of every available capability.
• Separate read, write, administrative, and destructive permissions.
• Require human confirmation for high-impact actions.
• Use allowlists for destinations, operations, and resource types.
• Apply rate limits, quotas, and maximum payload sizes.
• Treat retrieved text and tool output as data, not as instructions that can override system rules.

Observability

Record a correlation ID, model request ID when available, selected tool, validation outcome, latency, retry count, and final status. Redact or hash sensitive values according to your application’s requirements. Build dashboards or alerts around repeated malformed calls, rising timeouts, authorization denials, duplicate side effects, and unexpected tool-selection patterns.

For a broader evaluation approach, see LLM Prompt Testing and Prompt Engineering Evaluation.

Common mistakes

• Overly broad tools: A tool such as run_sql or send_any_request creates an unnecessary security boundary. Replace it with narrow, purpose-built operations.
• Trusting the model’s permission decision: The model can suggest an action but cannot establish identity or authorization.
• Retrying every failure: Retries can duplicate writes and amplify outages. Classify failures before retrying.
• Returning excessive context: Large tool results increase cost, distract the model, and may expose private data. Select and summarize fields server-side.
• Allowing unbounded loops: A repeated tool call can create latency and cost problems. Enforce round, time, and token limits.
• Skipping confirmation: A plausible interpretation is not proof that a user intended an irreversible action.
• Testing only the happy path: Include malformed JSON, missing fields, wrong resource ownership, expired credentials, slow dependencies, duplicate requests, prompt injection in retrieved content, and partial failures.

A useful test matrix includes the tool scenario, expected validation result, authorization outcome, retry policy, user-facing response, and log fields. Run it against each supported model or provider configuration; tool-selection behavior can vary even when the schema is unchanged.

When to revisit

Review this implementation whenever you add a tool, change an API contract, modify permissions, introduce a new model, or investigate a failed tool call. Also revisit it during scheduled quarterly reviews and before seasonal planning cycles that may change traffic, staffing, approval rules, or connected systems.

Before releasing an update, work through this short checklist:

1. Compare the tool schema with the actual server function and downstream API contract.
2. Run valid, invalid, ambiguous, unauthorized, duplicate, and timeout cases.
3. Confirm that confirmation prompts appear for every designated high-impact action.
4. Check that retries are bounded and safe for each operation.
5. Review logs for sensitive data, missing correlation IDs, and unclear failure categories.
6. Verify that the current model API and SDK support the request format you are deploying.
7. Document the tool owner, permission boundary, rollback method, and known limitations.

Function calling becomes dependable when the model’s role is narrow, the tool contract is explicit, and the application controls execution. Treat schemas, permissions, errors, and tests as one system rather than separate prompt-engineering tasks. That approach makes AI agent tools easier to review, troubleshoot, and evolve as your LLM application grows.