LLM Tool Calling: Connect Models to Your Systems

A language model on its own can only produce text. To look up a customer, create a task or check an order status, it needs a way to ask your systems to do something. LLM tool calling provides that bridge: you describe a set of functions the model may use, the model replies with a structured request to call one, your application runs it and the result goes back to the model. This is the mechanism behind most AI agents that act on business data. It is also where most of the risk sits, because the model is now choosing actions. This guide explains how tool calling works, how to design tools the model uses well and how to execute calls safely.

How LLM tool calling works

The OpenAI documentation describes tool calling as a multi-step conversation between your application and a model, with five high-level steps: make a request to the model with the tools it could call, receive a tool call from the model, execute code on the application side with input from the tool call, make a second request to the model with the tool output, and receive a final response or more tool calls (OpenAI: Function calling). Tools are defined with a JSON schema that describes their inputs.

Anthropic's documentation describes the same pattern for Claude: you pass a tool with an input schema, then execute the call when the model returns a tool use block. It distinguishes client tools, which your application executes, from server tools, which Anthropic executes (Claude: Tool use).

The key point in both cases: the model never runs anything itself. It proposes a call with arguments. Your code decides whether and how to execute it. That decision point is where safety controls belong.

Design tools the model can use well

How tools are defined affects how reliably the model picks the right one with the right arguments. The OpenAI guide recommends clear and detailed function names, parameter descriptions and instructions, explaining the purpose of each function and parameter and what the output represents, and using the system prompt to describe when and when not to use each one. Practical habits:

• Name tools after business actions, such as find_customer_by_email or create_follow_up_task, not internal endpoints
• Describe each parameter's format, with examples, such as a date format or an allowed list of values
• Use enumerations for fields with a fixed set of values
• Return compact, relevant results rather than whole records, so the model is not distracted and costs stay down
• Return clear error messages the model can act on, such as "no customer found with that email"

Keep the tool set small

More tools mean more chances for the model to choose wrongly. The OpenAI guide suggests keeping the number of initially available functions small for higher accuracy, evaluating performance with different numbers of functions and, as a soft suggestion, aiming for fewer than 20 available at the start of a turn. Combine functions that are always called together, remove tools that are rarely needed and consider separate agents or flows for unrelated jobs. Our guide to multi-agent workflows discusses when splitting work across agents helps and when it adds needless complexity.

Validate before executing

Treat every tool call as untrusted input, because it is generated text that may be wrong or may have been influenced by content the model read. Before running a call:

• Validate arguments against the schema and against business rules, such as an amount within limits or a record the user is allowed to access
• Check the user's permissions, not just the agent's; the agent should never be able to do more than the person it acts for
• Require human approval for actions that send messages, move money, delete data or change many records
• Apply rate limits and a maximum number of tool calls per conversation, so a loop cannot run away

Our guide to AI agent guardrails covers these controls in more depth, including defences against prompt injection.

Worked example: a hypothetical tool set

The table below sketches a hypothetical tool set for a sales assistant that works with a CRM. It illustrates how to classify tools by risk, not a tested design.

ToolActionRiskControl before executionfind_customerRead customer by email or nameLowLimit to records the user can seeget_open_dealsRead deals for one customerLowSame visibility rules as the CRMcreate_follow_up_taskCreate a task for the userMediumValidate date and owner; log the callupdate_deal_stageChange a deal stageMediumAllowed stages only; confirm with usersend_email_to_customerSend an emailHighShow draft; require user approvalbulk_update_recordsChange many recordsHighNot offered to the agent

Execute calls safely

When your application runs a tool:

• Use a service account or token with only the permissions the tool needs
• Set timeouts on every downstream call
• Make write actions idempotent where possible, so a retried call does not create duplicates
• Return a structured result to the model, including whether the action succeeded
• Never pass raw secrets or unrelated sensitive data back to the model

Decide what happens when a tool fails

Tools will fail: a CRM times out, a record is locked, a required field is missing. Decide in advance how the agent should respond. Return the error to the model in a form it can explain to the user, such as "the CRM did not respond; please try again later", rather than letting it guess. Do not let the model retry the same failing call indefinitely; cap retries in your code. For write actions, make sure a failure part-way through leaves data in a known state, and tell the user clearly what did and did not happen.

Log everything

Record each conversation's tool calls: the tool, arguments, who the agent was acting for, whether the call was approved, the result and the time taken. These logs are essential for debugging wrong answers, investigating incidents and showing auditors what the agent did. They also provide real examples for testing changes to prompts and tools.

Test tool use like any integration

Build a set of test conversations that should trigger each tool, and some that should not trigger any. Check that the model picks the right tool, passes valid arguments and handles errors well. Run these tests whenever you change prompts, tool definitions or the model version. Expect some variation between runs, and judge results across a set of cases rather than one.

Frequently Asked Questions

What is LLM tool calling?

It is a way for a language model to request that your application run a function, such as looking up a record, by returning a structured call with arguments. Your code executes it and sends the result back.

Does the model run the tools itself?

For tools you define, no. The model proposes a call, and your application decides whether to execute it. Some providers also offer server-side tools that they run.

How many tools should an agent have?

As few as the job needs. The OpenAI guide suggests keeping the initial set small for accuracy, with fewer than 20 as a soft suggestion.

Which tool calls need human approval?

Actions that send messages, move money, delete data or change many records should usually be approved by a person before they run.

Connect AI to your systems safely

We design tool sets, permissions, approvals and logging for AI agents that work with CRM and business data. See our AI agent orchestration and RAG service, or talk to us about LLM tool calling.

About the Author

Manav Sharma is the founder of Insyrge and a CRM solutions architect. He works across Zoho, Salesforce, HubSpot, Odoo and more, designing the CRM setups, integrations and automations that keep sales and operations teams working from the same data.