Give the Assistant Tools, Not Unlimited Autonomy

¶When is a fixed workflow safer and cheaper than an agent loop?

A study assistant that can choose its own actions can also repeat searches, read the wrong course’s notes, or spend your API budget without improving its answer. Before giving it that freedom, you need to decide which decisions genuinely require a model and which belong in application code.

I start with a fixed workflow for questions about course material. The application retrieves permitted notes, optionally reranks them, generates an answer, and checks its citations. A fixed workflow means code owns the sequence and its allowed branches; it does not mean every request follows an identical path. An agent loop instead lets the model repeatedly choose an action, inspect its result, and decide what to do next. The expensive difference is not tool access. It is who controls the next step.

¶Keep predictable decisions in code

Consider the illustrative request, “Compare self-attention with cross-attention using my lecture notes.” Your application might call search_notes with both terms, retrieve 8 passages, use rerank_passages to retain 4, and pass those passages to generate_answer. Then validate_citations checks that cited passage identifiers belong to the retrieved set. These are proposed application function names, not library APIs.

If retrieval returns no usable evidence, the workflow abstains. If a citation points outside that set, it returns an application-defined unsupported_citation failure rather than displaying an apparently sourced answer. This identifier check does not establish that the passage supports the claim; the grounding evaluation from earlier chapters still matters. Unless the task requires another information source, asking the model what to do next adds a decision without adding useful capability.

A fixed workflow also makes failures local. A retrieval timeout can produce retrieval_timeout and a retryable response instead of being interpreted as “no relevant notes exist.” A denied course lookup can produce access_denied; neither architecture should let the model override that denial. I would keep authorization inside the tool implementation, because a planner’s good intentions are not an access-control boundary.

¶Count decisions, not agent labels

Suppose the fixed path needs 1 generation request, while an illustrative agent execution needs 3 planning requests and 1 final-answer request. That is 4 model requests rather than 1, not proof of a fourfold bill: each request has different input and output lengths. Repeatedly appending tool results can increase later input costs, unless the application trims the history. Sequential decisions also add waiting time. Measure token totals and elapsed time on the same evaluation questions; request count alone cannot establish either cost or speed.

The uncomfortable point is that a capable model can make an unnecessary loop look productive. Rephrasing the same search 3 times produces activity, not necessarily additional evidence. I would require a measured improvement in answer quality before accepting that extra work.

An agent loop earns its place when the next useful action depends on an intermediate result that you cannot reasonably encode as a small set of branches. For example, comparing conflicting explanations across several permitted courses may require inspecting one result before choosing another source. Even then, autonomy must stay bounded; validation and termination controls follow in the next sections. MCP will supply an integration protocol later in this chapter, not permission to bypass those controls.

As a design exercise, take 10 study questions and write the next action after each possible retrieval outcome. If the same few branches handle them, keep the workflow. For the exceptions, what information must the model observe before it can choose a better next action than your code?

¶How does a model request a tool while Python retains execution authority?

A generated function name must never become permission to execute arbitrary Python. If the study assistant invents delete_notes, your application should reject it, not search for something callable with that name. I treat every tool request as an untrusted proposal: the model chooses what to ask for; Python decides whether the request is valid, permitted, and executable.

A tool-enabled model receives descriptions of available operations and their argument schemas. Instead of answering immediately, it can return a structured request containing a tool name, arguments, and a call identifier. The application validates that request, runs an approved function, and returns a result associated with the same identifier. The model can then use the result to compose its answer. This is message exchange, not the model reaching into your interpreter.

¶Keep the dispatcher deliberately boring

I would start with 1 read-only operation, read_note, rather than a general filesystem or database tool. Its contract accepts exactly 1 argument, note_id. The allowed note identifiers come from the authenticated session, never from a model-generated claim about who the student is. Unless that authorization context comes from trusted application code, argument validation only makes an unauthorized request well-formed.

The following runnable example uses a mocked model request and an application-owned envelope. Real SDK field names differ; adapt their responses into this shape rather than assuming this is a universal API format. The 64-character identifier limit is an illustrative application rule, not a model limit.

import json

NOTES = {
    "attention-01": "Self-attention combines information across tokens.",
    "private-02": "Unreleased assessment notes.",
}


def read_note(note_id):
    return {"note_id": note_id, "text": NOTES[note_id]}


def execute_request(request, allowed_note_ids):
    call_id = request["call_id"]

    def reject(message):
        return {"call_id": call_id, "error": message}

    if request["name"] != "read_note":
        return reject("unknown_tool")

    try:
        arguments = json.loads(request["arguments"])
    except (json.JSONDecodeError, TypeError):
        return reject("invalid_arguments")

    if not isinstance(arguments, dict) or set(arguments) != {"note_id"}:
        return reject("invalid_arguments")

    note_id = arguments["note_id"]
    if not isinstance(note_id, str) or not 1 <= len(note_id) <= 64:
        return reject("invalid_arguments")

    if note_id not in allowed_note_ids:
        return reject("access_denied")

    if note_id not in NOTES:
        return reject("note_not_found")

    return {"call_id": call_id, "result": read_note(note_id)}


request = {
    "call_id": "call-001",
    "name": "read_note",
    "arguments": json.dumps({"note_id": "attention-01"}),
}
allowed_note_ids = frozenset({"attention-01"})

print(json.dumps(execute_request(request, allowed_note_ids), indent=2))

for name, arguments, expected in [
    ("delete_notes", {}, "unknown_tool"),
    ("read_note", {"note_id": 7}, "invalid_arguments"),
    ("read_note", {"note_id": "private-02"}, "access_denied"),
]:
    rejected = execute_request(
        {
            "call_id": "call-test",
            "name": name,
            "arguments": json.dumps(arguments),
        },
        allowed_note_ids,
    )
    assert rejected["error"] == expected

The fixture assumes the outer envelope already contains call_id, name, and arguments; your SDK adapter must check that boundary too, otherwise missing fields raise KeyError. Inside the dispatcher, malformed JSON produces invalid_arguments, an unregistered operation produces unknown_tool, and an inaccessible note produces access_denied. Authorization runs before the existence check so that an unauthorized caller cannot distinguish a private note from a nonexistent one.

Notice the explicit function call. I avoid dynamic dispatch through eval or unrestricted getattr; saving a few lines is not worth exposing your Python namespace. When the tool set grows, use an explicit mapping of approved names to callables, while retaining operation-specific validation and permissions.

¶Return evidence, not new instructions

After execution, append the model’s tool request and the corresponding tool result using your provider’s required message structure. Preserve the call identifier: if a response contains 2 requests, their results must not become interchangeable. Returning a result without its matching request can trigger a provider validation error or leave the conversation incorrectly associated, depending on the API.

A successful read proves only that authorized data was retrieved. It does not make the note’s contents trusted instructions. Keep source identifiers with the returned text, and apply the grounding controls from the RAG chapters. Likewise, schema-constrained generation can reduce malformed arguments, but it cannot establish ownership of a note.

For an interview, explain why a valid request can still be denied: syntax describes the proposed action; authorization decides whether this user may perform it. As an exercise, add an extra argument to the fixture and verify that execution is rejected. Once rejection works, the next question is harder: how many opportunities should the assistant get to try again?

¶Can the assistant search notes and calculate a study schedule through validated tools?

A study assistant that retrieves the right chapter but invents a timetable is still unreliable. “Study for 120 minutes across 3 days” should produce an allocation whose sum is exactly 120, not a plausible-looking paragraph. I would give the model two narrow tools: search_notes for evidence and calculate_schedule for arithmetic. I would not expose a general Python evaluator; executing generated expressions costs more security work than this task deserves.

The tool contract should separate facts from assumptions. Search returns source identifiers and excerpts. Scheduling receives a total duration, a number of days, and a daily limit; it does not infer how long learning attention will take. Unless the student supplies that estimate, the assistant must ask or explicitly propose one. Correct arithmetic cannot validate an invented workload.

¶Keep the executable surface small

This standalone fixture uses 2 short, original notes and lexical matching so you can test the boundary without an API key. In the application, replace that matching block with the retriever already built, preserving permission filtering and source metadata. The application supplies allowed_ids from the authenticated session—not from model arguments. Filtering after retrieval is insufficient if unauthorized excerpts have already reached the model.

NOTES = (
    {"source_id": "notes-1", "page": 1,
     "text": "Attention assigns weights to token representations."},
    {"source_id": "notes-2", "page": 2,
     "text": "Retrieval supplies course evidence for an answer."},
)


def require_integer(value, name, lower, upper):
    if type(value) is not int or not lower <= value <= upper:
        raise ValueError(f"{name} must be an integer in [{lower}, {upper}]")
    return value


def execute_tool(name, arguments, *, allowed_ids):
    try:
        if type(name) is not str:
            raise ValueError("tool name must be a string")
        if type(arguments) is not dict:
            raise ValueError("arguments must be an object")

        if name == "search_notes":
            if set(arguments) != {"query", "limit"}:
                raise ValueError("search_notes requires exactly query and limit")
            query = arguments["query"]
            if (type(query) is not str
                    or not 1 <= len(query.strip()) <= 200):
                raise ValueError("query must contain 1 to 200 characters")
            limit = require_integer(arguments["limit"], "limit", 1, 5)
            terms = query.lower().split()
            matches = [
                {**note, "text": note["text"][:400]}
                for note in NOTES
                if note["source_id"] in allowed_ids
                and any(term in note["text"].lower() for term in terms)
            ]
            return {"ok": True, "matches": matches[:limit]}

        if name == "calculate_schedule":
            if set(arguments) != {"total_minutes", "days", "daily_limit"}:
                raise ValueError(
                    "calculate_schedule requires exactly "
                    "total_minutes, days and daily_limit"
                )
            total = require_integer(
                arguments["total_minutes"], "total_minutes", 1, 10080
            )
            days = require_integer(arguments["days"], "days", 1, 30)
            daily_limit = require_integer(
                arguments["daily_limit"], "daily_limit", 1, 480
            )
            if total > days * daily_limit:
                raise ValueError("requested workload exceeds daily capacity")
            base, extra = divmod(total, days)
            minutes = [base + int(day < extra) for day in range(days)]
            return {
                "ok": True,
                "total_minutes": total,
                "minutes_by_day": minutes,
            }

        raise ValueError("unknown tool")
    except ValueError as error:
        return {"ok": False, "error": str(error)}


permissions = frozenset({"notes-1"})
print(execute_tool(
    "search_notes", {"query": "attention", "limit": 2},
    allowed_ids=permissions,
))
schedule = execute_tool(
    "calculate_schedule",
    {"total_minutes": 120, "days": 3, "daily_limit": 60},
    allowed_ids=permissions,
)
assert schedule["minutes_by_day"] == [40, 40, 40]
assert sum(schedule["minutes_by_day"]) == 120

infeasible = execute_tool(
    "calculate_schedule",
    {"total_minutes": 200, "days": 3, "daily_limit": 60},
    allowed_ids=permissions,
)
assert infeasible["error"] == "requested workload exceeds daily capacity"
print(schedule)
print(infeasible)

The limits are application policy, not model capabilities: queries stop at 200 characters, results at 5 excerpts, and schedules at 30 days. Each excerpt stops at 400 characters. These bounds constrain returned content; they do not impose a retrieval timeout. A production retriever still needs its own deadline. Keep the exposed JSON Schema consistent with these checks, but retain runtime validation even when the provider supports strict structured arguments.

Notice type(value) is int. Python treats booleans as integers for some checks; isinstance(True, int) returns true. A model sending true for days must receive a validation error, not a one-day schedule. Unexpected fields fail too: adding user_id produces search_notes requires exactly query and limit, rather than silently accepting an apparent authorization override.

For the illustrative request, “Find my attention notes and divide 120 minutes over 3 days, at most 60 minutes daily,” the dispatcher returns the authorized excerpt and [40, 40, 40]. The assistant can cite the note and explain the allocation. If the request changes to 200 minutes, requested workload exceeds daily capacity is the correct result. Do not silently raise the limit; ask the student to change the workload, days, or cap.

¶Test the boundary, not just the happy path

I prefer deterministic tests here to judging whether the final paragraph sounds helpful. Add assertions that an unknown name returns unknown tool, a Boolean duration fails, and a search with no authorized matches returns an empty list. Empty evidence is not an execution error, but it must not become a fabricated citation.

As an exercise, try 121 minutes across 3 days. The allocation should be 41, 40, and 40, provided the daily limit permits 41. Then ask: if a tool rejects the request, how many opportunities should the model get to repair it before the application stops?

¶What do step limits, explicit state, approvals, and duplicate-action protection prevent?

A tool call can be valid and still be the wrong action to execute again. Your study assistant might search the same notes indefinitely, resume a cancelled task, or repeat a calendar write after a timeout. Schema validation prevents none of these. I put execution limits in Python, outside the model’s control; a prompt saying “stop after a few attempts” is not a limit.

¶Bound the work, not just the conversation

Start with illustrative limits of 6 model turns, 8 tool executions, and a 20-second elapsed-time budget per request. These are design choices, not measured recommendations. Count each tool call separately: 1 model response requesting 5 tools consumes 5 executions. Count failed attempts too, unless you deliberately allocate a separate retry allowance. Otherwise repeated errors become free work.

Check budgets before dispatch, and reserve capacity before starting parallel calls. A deadline check cannot interrupt an already blocked request; configure timeouts on the underlying clients as well. When a limit is reached, record budget_exhausted and return the evidence already collected, or an honest incomplete result. I prefer an unfinished answer to an invisible loop that keeps spending money.

¶Make the current state authoritative

A transcript is useful evidence, but it should not decide whether execution is permitted. Keep application-owned fields such as status, tool_attempts, and pending_action. Explicit state means recording the workflow’s current condition and allowing only specified transitions. For example, waiting_approval must not silently become running because another model response arrived.

This runnable guard demonstrates the check-before-dispatch rule. Its counter is local to one process; it is not a durable recovery mechanism.

from dataclasses import dataclass

@dataclass
class Run:
    status: str = "running"
    tool_attempts: int = 0
    max_tool_attempts: int = 2

    def reserve_tool_attempt(self) -> None:
        if self.status != "running":
            raise RuntimeError("run_not_running")
        if self.tool_attempts >= self.max_tool_attempts:
            self.status = "budget_exhausted"
            raise RuntimeError("tool_budget_exhausted")
        self.tool_attempts += 1

run = Run()
for _ in range(3):
    try:
        run.reserve_tool_attempt()
        print("Dispatch permitted")
    except RuntimeError as error:
        print(str(error))

Persist transitions before allowing a task to resume after a restart. Otherwise a cancelled run can return as runnable work. Keep cancelled terminal unless the user explicitly creates a new request.

¶Approve the exact action

Searching permitted notes and calculating a schedule do not justify adding calendar events. If you later add that capability, pause before the write. Show the destination calendar, dates, timezone, and event contents. Record approval against the authenticated user and an immutable action version, not a vague “yes” in the transcript.

I require fresh approval if those arguments change. Recheck authorization at execution time, even when approval exists: permission may have been revoked while the assistant waited. A mismatch should produce approval_required, not an attempt to persuade the model to remember the earlier terms.

¶Distinguish retries from new intentions

A timeout does not prove a write failed. The calendar service may have created the event before its response disappeared. Retrying blindly can create 2 events for 1 approved action.

Assign an application-owned action_id to the approved operation and reuse it across retries. Duplicate-action protection, often called idempotency, means repeated submissions have the effect of one submission. Use the destination’s idempotency support when available. A local “completed” flag alone cannot close the crash window between the remote write and recording success.

Without destination support or reliable reconciliation, record outcome_unknown and stop automatic retries. I would not promise exactly-once effects across that boundary.

Exercise: simulate a successful write whose response is lost. Does your assistant reconcile the existing action, pause for investigation, or incorrectly create another one?

¶How do MCP hosts, clients, servers, tools, resources, and prompts fit together?

Connecting the study assistant to a notes service should not require rewriting its tool loop for every new service. But replacing custom integrations with MCP does not make those services trustworthy. A malicious instruction returned through a standard protocol is still a malicious instruction. I would adopt Model Context Protocol (MCP) to standardize integration, not to delegate authorization or planning.

¶The host owns the application; the client owns a connection

The host is the AI application: in this project, the study assistant that manages the conversation, model access, permissions, and bounded execution loop. Inside that host, an MCP client maintains a connection to an MCP server. The server exposes capabilities backed by some system, such as the course-notes index.

Think of 1 host connected to 2 servers: a notes server and a calendar server. The host manages 2 corresponding clients, each maintaining its own server session. “Client” here does not mean the student’s browser, and “server” does not necessarily mean a remote machine. A server can run as a local process or as a remote service.

Connection setup includes protocol initialization and capability negotiation through initialize. Only use a capability when the connected server supports it; installing an MCP-compatible application does not guarantee that every server exposes every feature. An incompatible protocol version or failed initialization should stop that connection, not trigger a fallback that silently bypasses validation.

¶Three primitives, three different jobs

MCP separates what a server offers into 3 useful categories. Their usual control patterns describe interaction design, not security guarantees.

• Tools perform operations. A notes server might expose search_notes, accepting a query and returning matching passages. The client discovers tools through tools/list and invokes one through tools/call. Tools are typically model-selected, but the host still decides whether a requested call may execute.
• Resources expose readable context. A course outline could have the URI notes://courses/cs301/outline. The application can discover resources through resources/list and retrieve one through resources/read. Resources are typically application-controlled: the host chooses what context to retrieve and include. A URI is an address, not proof of permission.
• Prompts expose reusable interaction templates. A server might offer quiz_from_notes, with a course identifier as an argument. Clients discover these through prompts/list and obtain a selected template through prompts/get. Prompts are typically user-invoked; they are not a new, higher-priority instruction channel.

I would expose the existing search operation as a tool first. A resource fits a known document; a tool fits a parameterized search. Adding a prompt is worthwhile only when students need a reusable interaction, not merely because the protocol supports one.

¶Follow one request across the boundary

Suppose a student asks for evidence about database normalization. The host presents the permitted search tool to the model. The model requests search_notes; the host applies the validation and step budget already established, then its client sends tools/call. The server checks the authenticated student’s course access, searches the permitted notes, and returns passages with source identifiers. The host treats those passages as evidence, not instructions.

If the server returns another student’s notes, the integration has a data leak even though every protocol message is valid. If search reports a tool execution error, the host must not turn that error into an empty, apparently successful search. If the connection times out, the existing bounded-retry policy still applies; MCP does not decide whether repeating an operation is safe.

For an exercise, draw this request with 2 separate permission checks: the host’s decision to allow the tool and the server’s decision to allow the data. Then ask: if the model supplies a different course identifier, which component prevents the cross-course read?

¶Which task-success and recovery tests show whether the agent earned its extra cost?

An agent that eventually answers correctly after 6 tool calls may be worse than a fixed workflow that answers after 2. A convincing demonstration hides that difference: you see the final answer, not the failed searches, repeated requests, or permission boundary the model tried to cross. I would keep the fixed workflow as the default until a paired evaluation shows what the extra decisions buy.

¶Score the task, not the conversation

For the study assistant, create 30 held-out tasks: 10 ordinary notes questions, 10 questions requiring both evidence and schedule calculation, and 10 failure or permission scenarios. This is a starter suite, not enough evidence for a broad reliability claim. Run both implementations against the same document snapshot, permissions, model version, and task inputs. Repeat each task 3 times if model variability could change the result; keep repeated runs grouped by task when reporting uncertainty.

Define success before running anything. For “Use my revision notes to divide 180 minutes between three topics,” require supported topic selection, valid citations, allocations totaling 180 minutes, and no unauthorized reads. An eloquent answer with invented citations fails. Conversely, a refusal passes a forbidden-access test but fails an authorized scheduling task. I avoid a single “helpfulness” score here because it lets fluent failures conceal broken execution.

Record task_success, policy_violation, tool_calls, elapsed_seconds, cost_usd, and termination_reason for every run. Success requires both the requested outcome and compliance with the execution constraints. Preserve the tool sequence as evidence, with sensitive arguments redacted.

¶Inject failures at specific boundaries

Use deterministic tool fixtures before paying for live trials. The labels below are application-owned fixture outcomes, not prescribed MCP error names. Assert observable behavior rather than an exact sequence of model sentences.

Separate recovery from safe termination. A transient timeout is recovered only when the authorized task subsequently succeeds. An access denial is handled correctly when the assistant stops safely; obtaining the forbidden document is not recovery. Measure recovery rate only over deliberately recoverable cases, and report safe termination separately. Otherwise, adding easy refusals can inflate your apparent recovery score.

¶Charge failures to the agent too

Compute cost per successful task as total run cost divided by successful tasks, including money spent on failures. Keep task categories separate as well: an agent might justify its cost for scheduling while adding nothing to ordinary retrieval.

For an illustrative calculation—not a benchmark—suppose the fixed workflow costs $0.30 across 30 tasks and passes 21, while the agent costs $0.60 and passes 24. Cost per success rises from approximately $0.014 to $0.025; the 3 additional successes cost $0.10 each in aggregate. That calculation is meaningful only if the safety checks also pass. I would reject a candidate with an observed unauthorized read regardless of its average success score; zero observed violations still does not prove safety.

As an exercise, make the timeout permanent rather than transient. Does the assistant stop within budget and acknowledge missing evidence, or claim success anyway? The next chapter needs those tested termination rules and per-task measurements as inputs to deployment—not another polished demonstration.