Your Agent Reads the Whole Menu Before It Orders

Before your agent reads your question, it reads your menu. Every tool you have handed it — the name, the description, every parameter schema — gets loaded into the model's context window on every single turn, whether the model calls that tool or not.

GitHub's official MCP server is the example that keeps coming up: 93 tools and roughly 55,000 tokens of tool definitions. On a 200K context window, that is more than a quarter of the room occupied before the agent has done one useful thing. AWS published its own version of this in July 2026, and the numbers line up: a typical tool schema runs 550 to 1,400 tokens, and a coding session with five to ten tool servers installed can carry 50,000 to 67,000 tokens of definitions before you type your first prompt.

Two definitions, because everything below depends on them. MCP (Model Context Protocol) is a standard way to hand a model a set of tools — functions it can call, described in JSON. The context window is the fixed amount of text the model can hold at once. Your instructions, the tool list, the conversation, and every result all compete for the same space. Nothing is free and nothing is evicted quietly.

Rent, not a purchase

Most people think of a tool definition as documentation. It is really a fixture bolted to the floor of a small room you have to work in.

Split your agent's token usage into two buckets. The fixed cost is the system prompt plus every tool schema. The variable cost is reasoning, tool results, and retries. The fixed cost is the one you decide at design time, and it is the one that never shows up in a latency graph, because it does not slow any single call down. It just makes every call slightly more crowded.

Do the arithmetic on your own server before you argue about it:

• Count your tools. Say it is 15.
• Serialize their definitions to JSON and tokenize them with the model's tokenizer. Say you get 13,000 tokens.
• Multiply by turns per task. A task that takes eight turns pays 104,000 tokens of tool definitions. Not calls — definitions. The same text, eight times.

That is the rent. It is charged whether the agent is doing brilliant work or flailing.

The expensive part is not the schema. It is the guessing.

A bloated tool list has a second cost that is larger and harder to see. When many tools look alike — describe_volumes, describe_addresses, describe_snapshots, all thin, all similar — the model picks wrong. It passes a bad parameter, gets an empty result, and tries again.

In one measured comparison, an agent asked a single question against a 15-tool server needed seven tool calls to answer. The same question against a 4-tool version of the same server needed two. Same account, same data, same findings.

Five extra calls sounds like five extra calls. It is not. Each retry drops another result payload into the room, which shrinks the space left for the actual reasoning, which raises the chance of the next wrong choice. Bloat causes confusion, confusion causes retries, retries cause bloat. That is a feedback loop, not a linear tax, and it is the reason a tool list that felt fine in a demo starts feeling stupid in week three.

A menu, not a mirror

The root cause is almost always the same design reflex: take an API, wrap each endpoint as its own tool, ship thirty of them. It feels complete and honest. Every tool does exactly one thing. Unfortunately, the agent does not want one thing per endpoint — it wants one thing per question.

The rebuild that took that same AWS cost-triage server from 15 tools to 4 followed a different rule: design from the question the agent will actually ask. "Where am I wasting money in this account?" is one question with a handful of sub-checks. It maps to a small number of tools that each return a decision, not a payload.

In that rebuild the tool schemas shrank by about 88%, and the number of calls needed to answer the question fell from seven to two. That is the whole argument in one line: a smaller menu made the agent cheaper and also made it better.

The rules I would actually enforce in review

One tool, one question

If you cannot write down the sentence an agent would say before calling the tool, delete the tool. "Show me which storage volumes are unattached and costing money" is a question. "Calls the EC2 DescribeVolumes API and returns the complete response payload" is a mirror.

Return findings, not payloads

A raw cloud API response can carry forty fields. The agent reads six of them. Return the six, plus the one field that changes the decision — state, age, cost, whatever flips the answer. Response bloat is rent paid on data nobody reads, and it is paid on every follow-up turn because results stay in context.

Modes are parameters, not new tools

Four tools that each take a mode enum beat sixteen tools that each do one variant. But be honest about the trade-off: you have moved the complexity from the tool list into the parameter. If the enum values are unlabeled or overlapping, you have rebuilt the confusion spiral inside a single tool, which is worse, because now the model cannot even pick a different tool to recover.

Docstrings are prompt text, not documentation

Write tool descriptions for the model, not for a human reading a reference site. State when to use it, when not to use it, and what a partial or empty result means. Failure modes belong in the description, because the description is the only thing the model sees before it decides.

Delete tools nobody calls

Go through your traces. If a tool has not been called in a month of real usage, it is furniture in the room that nobody sits on. Removing it is free performance — and free correctness, because the model can no longer pick it by accident.

What lean costs you

Consolidation is not a free lunch, and anyone selling it as one has not shipped it.

• Discoverability drops. A single triage tool does not advertise that it also checks snapshots and load balancers. You pay for that with a longer, more careful description — which is still cheaper than four schemas, but not zero.
• Rarely used capability can be worth keeping fat. If a tool is called once a month for one specific check, and the alternative is a parameter soup that the model gets wrong, keep the small dedicated tool. Rent on a rarely used tool is cheap; wrong answers are not.
• You need evals for selection, not just output. Most agent test suites check whether the answer was right. Add a second metric: how many tool calls it took, and how many were retries. A correct answer after seven calls is a design smell you can measure.

Measure it in an afternoon

1. Dump every tool definition you register to a JSON file.
2. Tokenize it with the model's tokenizer. Do not trust an in-app context meter; different clients count differently and none of them count the same way twice.
3. Multiply by your average turns per task, then by tasks per day. That is your monthly bill for menus.
4. From your traces, count tool calls per question and retries per task. That is your ambiguity score.
5. Change one thing — merge, delete, or trim a payload — and run the same question set again. Compare both numbers, not just the answer.

The uncomfortable part is that lean tools require more design work up front. You have to know which questions your agent will actually ask, which means talking to the people who use it instead of reading the upstream API docs. That is the trade: an afternoon of thinking now, or a permanent tax on every turn for the life of the product.

The schema is the easy number to fix. The retries are the expensive one, and they only get fixed by the same move: fewer, clearer, question-shaped tools.

Key Takeaways

• Tool definitions load on every turn, used or not. 93 tools can eat over a quarter of a 200K window before the agent starts.
• Fixed cost (schemas) is visible and easy to measure. Variable cost (retries caused by ambiguity) is bigger and hides in traces.
• Wrapping each API endpoint as its own tool is the default anti-pattern. Design tools around the question the agent asks instead.
• One measured rebuild went from 15 tools to 4, cut schemas by ~88%, and dropped seven tool calls to two for the same question.
• Modes belong in parameters — but a sloppy enum just relocates the confusion inside one tool, where it is harder to escape.
• Track calls-per-question and retries-per-task as first-class metrics, alongside answer correctness.
• Delete tools nobody calls. Rarely used, high-variance checks may be worth keeping; everything else is furniture.