Session 10: Harness Engineering & Reliable AI Agents
Date: March 27, 2026
Duration: 1 hour 2 minutes
Presenter: Damien Li
Participants: 100+ attendees visible during the live session
Session Recording
📋 Overview
Session 10 explored Harness Engineering: the systems, tools, context controls, feedback loops, and operational safeguards that allow an AI model to complete long-running work reliably. Damien Li connected the progression from prompt engineering to context engineering and then harness engineering, arguing that model capability alone does not determine agent quality. The surrounding environment determines what the agent can see, which tools it can use, how it recovers from failure, and whether its work can be evaluated and improved.
The session combined industry analysis with a hands-on document-engineering demonstration. A coding agent analyzed PDF report templates, represented their layouts as structured YAML, generated Excel outputs, and refined the result through reusable Skills and feedback. The closing discussion expanded the same ideas to durable execution, identity and permissions, stateful sandboxes, observability, agentic reinforcement learning, and A2A applications.
🎯 Key Topics Covered
1. From Prompt Engineering to Harness Engineering
Presented by: Damien Li
The opening section described an evolution in how teams improve AI systems:
- Prompt Engineering: improve the instructions given to a model.
- Context Engineering: control the information, memory, and references available to the model.
- Harness Engineering: design the complete runtime around the model, including tools, workflow, state, evaluation, permissions, and recovery.
The agent's context window remains a practical bottleneck. Adding more context does not guarantee better retrieval, and long-context accuracy can decline as input grows. A harness should therefore expose the right information progressively instead of loading every possible document and tool at once.
Design principles discussed:
- Use progressive disclosure so detailed context appears only when needed.
- Prefer a small set of precise tools over a large, ambiguous tool catalog.
- Find the useful “sweet spot” between missing context and overloaded context.
- Separate durable state from short-lived conversational history.
- Make tool calls, decisions, failures, and retries visible to developers.
2. The Harness as a Dataset
The session highlighted the idea that execution trajectories become valuable data. A useful trajectory records more than the final answer: it captures what the agent observed, which tools it selected, the order of its actions, where it failed, and which feedback led to a better result.
This changes the harness from a wrapper around the model into part of the product's learning system. Teams can use real trajectories to:
- identify repeated failure modes and missing tools;
- compare prompts, models, and workflow variations;
- turn production incidents into regression evaluations;
- improve tool descriptions and context-retrieval policies;
- prepare higher-quality data for continual learning or agentic reinforcement learning.
The competitive advantage may therefore come from the combination of domain tasks, harness design, and feedback data rather than from access to a unique foundation model.
3. Document Engineering Demo
Damien demonstrated an agent workflow that converted business-report examples into reusable document definitions. The project contained source PDFs, Excel templates, YAML layout definitions, generated outputs, tests, scripts, architecture notes, and a dedicated document-engineering Skill.
Demonstrated workflow:
- Inspect a source PDF and existing report examples.
- Identify layout regions, labels, data placeholders, styles, and repeated structures.
- Represent the document as a structured YAML definition.
- Generate an Excel report template from the definition.
- Compare the result with the source and refine spacing, spans, formatting, and content.
- Capture the repeatable procedure in
SKILL.mdand validate it with tests.
The demo illustrated why a harness needs more than a single prompt. The agent benefited from file conventions, deterministic scripts, structured definitions, reference examples, validation steps, and a workspace where intermediate artifacts remained available across iterations.
4. Research → Plan → Execute → Verify
For complex tasks, the session recommended separating work into explicit phases and using subagents to isolate context. Research findings can be written to research.md; a reviewed implementation plan can be persisted to plan.md; execution then follows the approved plan with verification after each meaningful step.
Where human attention has the most leverage:
- clarifying the real objective and system constraints;
- reviewing research before it becomes an implementation assumption;
- approving the plan before large amounts of code are generated;
- defining evaluations and acceptance criteria;
- intervening when the agent reaches an ambiguous or consequential decision.
The session emphasized that human involvement is often more valuable during specification and planning than during a final line-by-line review. A wrong requirement or plan can create far more incorrect code than an isolated implementation mistake.
5. Agentic Reinforcement Learning and Feedback
Agentic reinforcement learning differs from a single-turn chatbot evaluation. An agent rollout may span many steps, call tools repeatedly, operate in a large action space, and receive a useful reward only after a long sequence completes.
Challenges discussed:
- long, multi-step trajectories with hundreds of possible tool calls;
- sparse rewards when difficult tasks succeed only occasionally;
- dependence on real environment signals such as tests, builds, and linters;
- the need to reproduce failures inside consistent sandboxes;
- separating model limitations from defects in prompts, tools, or the harness itself.
The session referenced coding-agent training as an example: models can learn deeper repository search, iterative lint fixes, self-written unit tests, and smaller changes when the training environment provides realistic tools and reliable feedback.
6. Infrastructure for Production Agents
Damien grouped several infrastructure concerns that become increasingly important as agents take on longer and more consequential work.
| Capability | Example discussed | Role in the harness |
|---|---|---|
| Durable execution | Temporal | Preserve workflow state and resume long-running tasks after process or network failure |
| Identity and permissions | Oasis Security | Give agents task-appropriate access while keeping actions traceable and controlled |
| Stateful sandbox | Daytona | Provide an isolated workspace that can persist across a longer agent workflow |
| Ephemeral code sandbox | E2B | Run untrusted or generated code inside a bounded execution environment |
| Observability and evaluation | Braintrust | Record traces, score results, compare changes, and turn failures into evaluation cases |
These layers complement rather than replace one another. A production harness may need workflow orchestration, secure identity, sandboxed compute, evaluation, and a coordination interface to work together as one system.
7. Agentic Engineering and A2A Applications
The closing section described agentic engineering as a broader discipline for coordinating models, tools, workflows, data, permissions, evaluations, and human decisions. The next generation of agent products may look less like a single “smarter chatbot” and more like an operational platform that supervises many specialized capabilities.
The session also visited projects from the Second Me A2A Global Hackathon, including applications where personal agents interacted in social, creative, and collaborative environments. An “Artful Forests” example showed agents contributing messages and memories to an evolving visual environment. These demos illustrated both the creative potential of agent-to-agent protocols and the need for clear identity, trust, and action boundaries.
📊 Presentations & Materials
Session Recording
🎥 Watch the March 27, 2026 recording
Recording highlights:
- Enterprise-agent report review and the rise of Harness Engineering
- Context-window limits and progressive disclosure
- PDF-to-Excel document engineering with Skills and YAML definitions
- Research, planning, implementation, and human-review checkpoints
- Agent trajectories as evaluation and training data
- Durable execution, permissions, sandboxes, and observability
- Second Me A2A hackathon examples and closing discussion
No standalone presentation files were supplied with the recording. The screenshots above were captured from the session for reference.
🔗 Featured Resources
Harness Design and Learning
- The Bitter Lesson - Rich Sutton's essay on general methods that scale with computation
- Temporal - Durable execution for long-running and failure-prone workflows
- Braintrust - Tracing, evaluation, and experimentation for AI applications
Security and Sandboxing
- Oasis Security - Identity and access management discussed for agentic environments
- Daytona - Stateful infrastructure for isolated development and agent workspaces
- E2B - Secure cloud sandboxes for AI-generated code and tools
Agent Ecosystems
- Second Me A2A Global Hackathon - Agent-to-agent projects demonstrated during the session
- Agent Skills in VS Code - Reusable instructions, scripts, and resources for coding agents
🎮 Quiz Activity
No structured quiz activity or scored quiz was included in the supplied recording.
🏆 Quiz Results & Winners
| Status | Details |
|---|---|
| No ranking published | The recording did not include a scored quiz or winner announcement. |
🔑 Key Insights
- The model is only one component: tools, context, state, permissions, evaluation, and recovery determine whether an agent is dependable.
- More context is not always better: progressive disclosure and focused retrieval protect the context window from noise.
- Trajectories are product data: intermediate decisions and failures reveal how to improve the harness and create meaningful evaluations.
- Planning has high leverage: human review before implementation can prevent entire branches of incorrect work.
- Real feedback makes agents learnable: tests, linters, builds, and stable sandboxes provide stronger signals than subjective output review alone.
- Production autonomy needs infrastructure: durable execution, least privilege, isolation, observability, and rollback are core requirements.
📚 Further Learning
For Developers
- Separate one complex agent task into research, plan, execute, and verify artifacts.
- Audit the tools exposed to an agent and remove overlapping or ambiguous options.
- Convert a repeated workflow into a Skill with deterministic scripts and validation steps.
For Teams
- Record agent trajectories and turn real failures into a regression evaluation set.
- Define permission boundaries and recovery procedures before granting broader autonomy.
- Measure the complete system—including model, harness, and feedback loop—rather than comparing models in isolation.
🙏 Acknowledgments
Special thanks to:
- Damien Li for the Harness Engineering analysis and document-engineering demonstration
- Aimee Li for hosting and facilitating the session
- All participants for the questions, discussion, and continued knowledge sharing
Design the environment, not only the prompt
A reliable agent needs focused context, precise tools, persistent state, executable feedback, and clear recovery paths. Improve those elements together as one harness.
Autonomy expands the security boundary
Long-running agents may accumulate credentials, files, state, and network access. Use least privilege, isolated execution, traceable actions, and tested rollback before increasing autonomy.
Session 10 | March 27, 2026 | Harness Engineering & Reliable AI Agents | Damien Li