Replacing custom AI tools with standard Unix commands caused a massive token spike because the Copilot agent engaged in open-ended browsing, flooding its context window with irrelevant codebase data instead of focusing on the pull request diff.
- Offline Benchmarking: Implement rigorous telemetry and tracing to observe the agent's exact execution paths and tool calls before making architectural changes.
- Workflow Rhythm Prompting: Rewrite system instructions to enforce a strict "ask, narrow, read, decide" sequence to prevent endless context gathering.
- Batch Cheap Discovery: Force the LLM to use glob and grep to find candidate files before ever calling view to read expensive file contents.
Tightly tuning system prompts to a specific workflow makes them highly brittle, creating an ongoing maintenance burden requiring rewrites whenever the underlying foundation model is updated.
Script
The Problem: Upgrading Tools Broke the AI
GitHub tried to upgrade their Copilot code review agent, and the whole thing fell apart. They took out the old, custom search tools and swapped in standard Unix-inspired shared tools.
- Grep
- Glob
- View
The exact commands we rely on every day. They expected a clean, easy win. Instead, token costs shot up. The number of useful review comments dropped. Upgrading to better tools actually made the AI perform significantly worse.
Picture this. You plug a standard set of search tools into your LLM agent. You expect a quick efficiency win. But suddenly your token costs spike. Why? Because your AI is reading half the codebase just to review a three-line pull request diff. It is spinning its wheels, burning your budget, and ignoring the actual task.
The Investigation: Why Standard Tools Failed
To understand why standard tools broke the AI's performance, you have to look at what Copilot was using before the upgrade. The original code review agent relied on custom code exploration tools.
- List directory
- Search file
- Read code
These were not thin wrappers around standard commands. When the agent asked for a specific line of code, these custom tools would automatically return the requested lines, plus a large chunk of surrounding code context. They were fat tools. They did the heavy lifting of context gathering because early agents were bad at knowing what else they needed to read.
But maintaining a custom toolset for every single AI feature is an architectural nightmare. GitHub wanted to consolidate. The Copilot CLI already used a shared harness of standard Unix tools. It used glob to find files. It used grep to search text. It used view to read files. No hidden context padding. Just clean, standard outputs. Migrating the code review agent to these shared tools looked like a simple one-to-one swap on paper.
But when GitHub ran the offline benchmarks, the agent’s behavior went completely off the rails. The benchmarking traces revealed a massive browsing loop. Generic tools trigger generic behavior. The instructions for these shared tools were originally tuned for a broad, interactive coding assistant. So the code review agent started acting like one. It would see a small pull request diff, but instead of focusing on the change, it would use glob to guess at likely file paths across the entire project. It would use view to read huge chunks of unrelated architecture. It would find a new symbol, use grep to find every instance of it in the repository, and carry all of that extra code forward.
When you or I review a pull request, we start at the diff. We ask highly targeted questions. Where is this function called? Is there a test with the same pattern? We do not open thirty random files to understand the entire repository architecture before approving a one-line bug fix. We want the minimal context needed to answer the question. The AI agent was trying to boil the ocean.
The generic tools implied an open-ended exploration workflow. Every single tool result is not a disposable printout. For an LLM agent, those results are extra tokens. They get permanently stuck in the working context window. Broad tools led to context bloat, which led to a distracted, expensive agent.
The Fix: Forcing a Strict Workflow Rhythm
The only reason the Copilot team caught this was because they had built a rigorous offline benchmarking and tracing harness. They could see the exact execution paths, the specific tool calls, the volume of data coming back, and where the errors happened. They could see that the agent was widening the search instead of narrowing toward evidence.
If you are building your own custom LLM agents, internal developer platforms, or automated review pipelines, this is your baseline requirement. You cannot successfully adopt constrained workflow patterns if you cannot observe the step-by-step behavior of your agent first. High setup cost is the entry fee for this kind of debugging. You need that telemetry.
The fix for GitHub was not to revert to the old custom tools. The Unix tools worked perfectly fine. The fix was rewriting the system instructions to explicitly encode a strict workflow rhythm. You cannot just bolt generic tools onto an agent. You have to force the agent to stop browsing. You give it a reviewer rhythm: ask, narrow, read, decide.
The new system prompt forced the agent to start strictly from the pull request diff. It instructed the agent to batch cheap discovery first.
- Use glob when the path is uncertain.
- Use grep to find candidate files or call sites.
- Do all of this before ever calling view to read a single file's contents.
Let's say a diff changes an authorization helper. The generic agent would try to view every file that imports the helper. The tuned agent is instructed to act differently.
- It starts at the helper.
- It greps for callers of that helper.
- It uses glob to find likely route or controller files.
- Then it calls view on only the most relevant caller ranges.
- Finally, it decides if the change introduces a risk.
They even had to explicitly script the error recovery. If a grep search failed, the agent's natural instinct was to start guessing neighboring paths with glob. That sparks another exploration loop. The new instructions told the agent exactly what to do instead: If grep fails, retry the failed grep with a simpler, escaped search string. Stop exploring. Start narrowing.
The Results (With Caveats)
By forcing the agent to manage its own context window intentionally, GitHub claims this prompting resulted in roughly twenty percent lower average review cost. But you have to read that metric carefully.
That twenty percent almost certainly refers strictly to LLM token consumption. It does not mean they saved twenty percent on total infrastructure compute. It definitely does not measure actual human developer time saved.
They also claim this tuning maintained the same review quality. Automated evaluation of AI code review quality is notoriously subjective. How they define this baseline or protect against false positives is glossed over in the reporting.
There are also glaring missing pieces in this architecture. Standard grep and view commands do not inherently respect LLM token limits. If a grep search returns one thousand matches, how does the system prevent a context window overflow? The engineering breakdown skips that mechanism entirely.
Then there is the ongoing maintenance burden. Prompts tuned this tightly to a highly specific workflow are brittle. They tend to degrade entirely when the underlying foundation model gets updated. You will be rewriting these instructions.
If you want to avoid that maintenance burden, you can look at off-the-shelf agent frameworks. Many still use those fat, context-packing tools. You might actually prefer them. They require significantly less fine-grained prompt tuning to get a baseline result out of the box. But the trade-off is efficiency. You will burn more tokens.
The Core Takeaway: Tools Dictate Behavior
The core takeaway for developers building agentic workflows is about control. Tools dictate agent behavior. Tools are not just implementation details you can swap out behind the scenes. They are the agent's sensory interface.
They dictate what the agent notices, how it searches, and when it decides it has enough evidence. The tool descriptions are essentially API documentation for the LLM. If your API docs are vague, the LLM makes bad decisions. If your agent is suffering from context bloat and burning tokens to gather irrelevant data, look at the tools. Look at the instructions attached to them.
If you are building a general-purpose coding assistant where open-ended discovery is the actual feature, broad tools make sense. But if you want a system to execute a highly specific task, your system prompt must dictate exactly how to handle the tools. Ask. Narrow. Read. Decide. Shared tools only scale when the instructions and the benchmarks match the specific job.
This is takeyourpills.tech. Go ship something.