|
How we built an agent harness that cuts token consumption by 90%
A practical guide to building with Jev: when to use code, when to call a model, and how to check its work.
|
| |||||||||||||||
 |
I think we’re building agents backwards. We give an agent a job. It plans, chooses tools, makes mistakes, recovers and finishes. Give it another version of the same job, and it can spend as much time figuring out how to do it.
The first time I do something new, I figure it out. By the tenth time, I’ve got a process. By the hundredth, much of that process is muscle memory. I still need judgment when something unfamiliar happens, but I’m no longer working out every step from scratch.
We’ve been experimenting with giving agents that same ability at Grep: let the agent figure out how to do the job, then capture what it learns as a structured workflow. Predictable steps run in code. Judgment calls go to a model. When the workflow encounters something it can’t handle, a general agent can investigate further.
Say you ask an agent to investigate 1,000 companies. Each company is different. Much of the process repeats: find the right entity, gather evidence, check it against your criteria and write up the result. Why should the agent have to rediscover that process 1,000 times?
We built a small domain-specific language, or DSL, for these workflows. In it, we specify which tools to use, how to branch, and when to retry or escalate.
We inspect each run for wasted steps and failures, then test proposed changes before using the workflow on more cases.
We save those changes in a workflow the next run can use.
Jev has been useful here. It lets us ask small, explicit questions about the evidence an agent collects, cheaply enough to use throughout a workflow. Does this source support the extracted fact? Which profile needs more research? Is there enough evidence to apply this rule?
Those decisions help the harness choose where to spend its time.
In one screening case, our original agent spent 51 minutes making 826 tool calls. The refined workflow reached the same decision in about three minutes with roughly 30 calls. Across a separate 100-alert comparison, model costs fell from $2.89 to $0.25 per alert.
I shared some of the results on X:
My cofounder Miguel wrote up the system, including the workflow changes and our cost comparisons.
Thanks for reading The Operator's Guide to AI! Subscribe for free to receive new posts and support my work.
