Harness from first principles
Introduction
Hey ! Everyone knows that the frontier is advancing at a very fast pace and we all can't stop talking about AI models.
- Which is smarter?
- Which is better?
- Which one codes better?
We also hear people talk about AI agents, but fundamentally... What are AI agents in the first place ?
A model alone, doesn't form an agent, but give the model tools, give it memory, context, an environment, and most importantly - a way to interact with your environment.
This new stacked up model, can not only generate text but now is capable of taking actions.
This model can now read files, call APIs, execute code, browse the web, interact with databases, and use its tools to accomplish a goal.
But then there's another question, what if our "agent" fails ? who stops it ? who manages its memory ? and most importantly, who has the power to control this agent ?
You can build all the systems down independently but eventually you need a system that brings them all together. This system is often thrown around as harness. And again, harnesses were nothing new, engineers have been using harnesses for decades, so how did we go from this to Claude Code spawning 5 subagents executing tasks in parallel ?
The idea of the "Harness"
When you look up Harness in the dictionary,
The gear or tackle, other than a yoke, with which a draft animal pulls a vehicle or implement.
Which is true, it literally refers to a piece of armor or equipement. However when you associate it with a horse, you feel the power in the word "harness" itself !
So, a harness came to be a way to control and direct that power.
The test Harness
Long before LLMs, software engineers have been using test harnesses for the machinery used to drive software while testing.
The test harness could provide inputs, control the software's execution, see traces, capture outputs and compare it with expected results.
This idea expanded beyond simple test harnesses, it became a way to create controlled environments around systems, simulating failures, controlling execution, observing results, supplying inputs, etc.
So the environment around the system - the harness, became very important and useful to improve the system itself.
Eval Harnesses
These harnesses soon were used to test and evaluate ML models, and they were designed to just
- Give inputs to the model
- Capture outputs from the model
- Compare the outputs with expected results
This became a very important part of the ML model lifecycle, and it was used to evaluate models on various metrics like accuracy, precision, recall, etc.
The Agentic Harness
This is the most current stage, the harness isn't there to test the model, it now helps you work with the model increasing its capabilities, and helping you build systems around it.
It helps you control the model and includes things like context management, tool execution, permissions, recovery, long sessions, subagents. Having loops, tools, and memory, the harness can now help you build systems that are more robust and reliable.
Why do we need an agentic harness ?
let us consider Qwen 3.8 27B GGUF model which can run locally on your machine.
You tell it to read a file, however it doesn't have the ability to read files, so it fails. You tell it to read a file again, and it fails again. You tell it to read a file again, and it fails again.
Then you realise it never had the read file tool, so you give it the read file tool. Now what happens is something interesting.
Let's check out its reasoning traces.
User asked me to read a file, I have access to the read file tool so let me use it to read the file.
read_file("file.txt")
I have read the file
Wait here and ask yourself a simple question, did it actually read the file ?
You might say yes, it says it read the file, read_file() so it must've, but
there was no reading of the file, the model just said that it called a tool,
but there was no system for it to capture the output of the tool, and see through its execution and pass it back to our model.
A model with tool descriptions is not an agent.
The need for the underlying system to manage the tool execution, capture the output, and pass it back to the model is what makes a harness so important.
That is one reason, but this was just one tool call, imagine having 10 tools, what if it fails or runs out of context ? Can the tasks be executed parallely ?
This is exactly why we need an agentic harness.