Waitlist open
Write a better harness by practicing.
AI engineering moved to the systems around the model. Same model for everyone. A real environment. A hidden eval. Leave your email. We will send a seat when the playground is up.
One email. We write when problem 01 opens. Nothing else.
How it works
Read the trace. Change the loop.
- 01
Run the starter
Three visible tasks. A naive ReAct loop. You will fail some of them on purpose.
- 02
Read a failing trace
System prompt, every tool call, every result, the grader verdict. This is how you debug a harness.
- 03
Edit and submit
Thirty hidden tasks. Same model. Pass is 85 percent. A careful loop lands higher, with tokens still in budget.
Get a seat when it opens.
No account yet. One email. The playground comes after the host is up.
FAQ
Questions
What is a harness?
The software around the model: the loop, the tools, the context, retries, when to stop. Not the model. That file is what you write here.
Why pin the model?
If everyone picks a different model, you cannot tell who built the better loop. Same model, same environment, same hidden eval. The only variable is the harness.
What is a trace?
The full transcript of one task: system prompt, every message, every tool call and result, the final answer, the grader verdict. You debug from this, not from the score.
When do I get in?
When the eval environment is hosted. We will email you a seat. No drip sequence. No newsletter.
What will I submit?
One Python file. from harness import llm, tools, Task. The starter is a naive ReAct loop. Run three visible tasks. Submit thirty hidden. Pass is 85 percent.
Is it free?
Yes for the waitlist and the first problems. A cheap pinned model. Run is three tasks. Submit is capped so nobody burns the bill.