How to Use Function Calling with Local Models for Workflow Automation

Function calling with a local model works much like it does with a hosted API, with one important difference: nothing executes on the inference server. The model reads the tool definitions you send, decides whether a tool is needed, and returns a structured request containing the tool name and its arguments. Your application runs the matching function, then sends the result back for the model to use in its final answer. That division of labour is what makes local function calling practical for workflow automation: the model handles interpretation and routing, while your code keeps control over credentials, side effects, and data.

This guide is aimed at technically curious beginners and intermediate practitioners who already run a local model and want to connect it to real actions. It covers the execution loop, a concrete tool schema, reliability and security concerns, and the failure modes that appear most often in practice.

What function calling is — and what it is not

Function calling, also called tool calling, does not give a language model the ability to run code. The model’s only output is text, and when it “calls a tool” it is producing a structured message instead of a prose answer. Microsoft’s Foundry Local Lab describes the pattern plainly: the model recognises when a tool would help and returns a structured request for your code to execute; your application runs the function, sends the result back, and the model incorporates that information into its final response.

The word “local” in local tool calling refers to where execution happens. Groq’s documentation defines it as tool execution that happens in your application code rather than on the inference provider’s servers. The functions you implement may still reach external resources such as databases, APIs, or internal services — they are “local” because they run on the same machine as your application, under your security and logging controls.

This is worth separating from remote MCP orchestration. Local tools can come from local MCP servers over stdio, and those servers supply both the definitions and the implementations, but your code is still responsible for orchestrating the calls. In the remote MCP pattern, the provider’s infrastructure handles the entire loop instead.

The four-stage execution loop

Every local tool-calling integration, from a two-tool script to a multi-step agent, reduces to the same cycle. The Foundry Local Lab breaks it into four stages:

Groq’s flow diagram adds one detail that matters for automation: after a tool result is returned, the model may issue more tool calls rather than finishing. Stage 4 therefore loops back to stage 2 until the model stops requesting tools. Your orchestrator, not the model, decides when that loop ends.

Prerequisites before writing orchestration code

• A model and runtime that support tool calling. Tool-calling support is a property of both the model and the serving stack. Check the model card and the runtime’s documentation for tool or function support before building anything on top of it. Heavily quantised builds of otherwise capable models sometimes degrade on structured output.
• A small, narrow set of functions. Start with one or two read-only tools. Broad tools with many optional parameters are harder for the model to select correctly and harder for you to validate.
• An argument validator. Treat every argument the model produces as untrusted input, even when the model is running on your own machine.
• A step limit. Decide the maximum number of tool round-trips before the run is aborted.

Building the loop, step by step

Step 1: Describe each tool with JSON Schema

The description field is the model’s only instruction for when to use a tool, so write it as a routing rule, not marketing copy.

{
  "type": "function",
  "function": {
    "name": "get_order_status",
    "description": "Look up the fulfilment status of a customer order using its order ID. Use this only when the user supplies or references a specific order.",
    "parameters": {
      "type": "object",
      "properties": {
        "order_id": {
          "type": "string",
          "description": "Order identifier, for example A-10231"
        }
      },
      "required": ["order_id"]
    }
  }
}

Step 2: Send the conversation and the tool definitions

The request contains the user’s message plus the array of tool schemas. If the model judges that no tool is needed, it answers in prose and the loop exits immediately — a useful property, because it means your automation does not pay for a tool round-trip on trivial requests.

Step 3: Parse the tool call and validate the arguments

The model returns a structured request with the tool name and arguments. Before dispatching, check that the name is in your allowlist and that the arguments satisfy the schema: required fields present, types correct, values within expected ranges. Reject anything else and return a short error message as the tool result so the model can correct itself.

Step 4: Execute the function in your own code

This is the step that is genuinely local. The function runs in your process, with your credentials, under your logging. Keep the execution surface as small as the task allows.

Step 5: Append the result and call the model again

Add the tool output to the conversation as a tool-role message tied to the originating call ID, then re-request. The model now has the data it asked for.

Step 6: Bound the loop

The following skeleton is illustrative pseudocode, not a tested implementation; field names differ between runtimes, so adapt them to your client library.

tools = [get_order_status_schema, create_draft_schema]
messages = [{"role": "user", "content": user_request}]

for step in range(MAX_STEPS):
    response = model.chat(messages=messages, tools=tools)
    if not response.tool_calls:
        return response.content            # final answer

    messages.append(response.message)      # assistant turn with tool_calls

    for call in response.tool_calls:
        args = validate(call.function.name, call.function.arguments)
        result = dispatch(call.function.name, args)   # your code
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result)
        })

raise RuntimeError("tool loop exceeded MAX_STEPS")

The observable success condition is simple: the loop exits with a text answer, and your logs show one entry per tool execution with its validated arguments.

An illustrative automation: triaging incoming content

The following is a design sketch, not a deployed case study. A watcher process picks up new submissions — for example, drafts arriving in a WordPress queue — and passes each one to a local model with three tools attached: classify_topic, find_similar_posts, and create_review_task. The model reads the submission, calls the classifier, calls the duplicate check with the returned topic, and then either creates a review task or returns a short explanation of why the item needs no action.

Two properties make this shape work well locally. First, the first two tools are read-only, so a malformed argument produces a wrong answer rather than a wrong write. Second, the only side-effecting tool, create_review_task, creates a task for a human rather than publishing anything. Where factual accuracy or publication decisions are involved, keeping a human checkpoint in the loop is the safer default — the model routes work, it does not approve it.

Reliability, security, and operational limits

• Validate, never evaluate. Arguments arrive as strings. Parse them against the schema and dispatch through a lookup table of known function names. Never pass model output into an interpreter, shell, or raw query.
• Treat tool output as untrusted. A retrieved document or API response can contain text that looks like an instruction. Because tool results are appended to the same conversation the model is reasoning over, that text can influence the next decision.
• Make write tools idempotent. Models occasionally repeat a call. An idempotency key or a uniqueness check prevents duplicate records.
• Watch context growth. Each round-trip adds the assistant’s tool-call message and the tool result to the conversation. Long tool outputs inflate the prompt and the KV cache, which affects both memory use and latency on local hardware.
• Expect smaller models to be less reliable at schema adherence. Malformed JSON and invented parameter names are more common on compact or aggressively quantised models. Validation plus a single corrective retry handles most of these cases.

Diagnosing common failures

• The model answers in prose instead of calling a tool. Usually a description problem: the tool’s purpose is too vague, or the model does not support tool calling in this runtime configuration. Tighten the description, then verify support in the model card and runtime docs.
• Arguments are malformed or missing. Validate, then return a concise error as the tool result — for example, a message stating which field was invalid — and let the model retry once.
• The loop never terminates. Cap the step count and detect repeated identical calls. Returning an explicit failure message as the tool result usually stops the repetition.
• The model ignores the tool result. Check that the result is appended with the correct role and the call ID that matches the originating request. A mismatched ID effectively hides the result.
• It works against a hosted API but not locally. Different runtimes apply different chat templates and tool-call parsers. A model that emits tool calls correctly under one template may emit plain text under another.

Where to start

Begin with a single read-only tool and log every request, argument set, and result. Once the loop exits cleanly and the logs are readable, add a second tool and a step limit, then introduce one side-effecting tool behind a human review checkpoint. The orchestration code you write is the same in all three cases — only the allowlist grows.

If you are working through this on your own setup, the community is a good place to compare notes: share your runtime, model, and quantisation, the tool schema you used, and the exact output or error you saw. Configuration differences between runtimes are the most common reason the same schema behaves differently on two machines.