钛媒体APP

wrote a column ·

The agent has entered the Harness-driven era

(The author of this article is from Xia Guang AI Lab, published by Titanium Media with authorization)

By Xia Guang AI Lab

Recently, a hot topic in the AI technology community is that Anthropic accidentally exposed the complete source code of its AI programming tool, Claude Code, with over 512,000 lines of code. While these leaked codes do not reveal any groundbreaking new algorithms, they fully expose the engineering practices of a leading vendor’s Agent.

On April 10, Zhu Zheqing, founder of Pokee.ai, joined an online closed-door session hosted by Jinqiu Fund's 'Deep Talk with Builders' to share insights on the topic 'Looking at Harness Engineering and current Post-training through the lens of the Claude Code leak.'

He believes that Anthropic’s architecture is highly tailored to the Claude model, and directly migrating it to other models would result in significantly reduced performance. However, itsHarness design philosophy, modular structure, and deep integration with post-trainingoffer extremely valuable lessons for self-developed Agents.

Over the past three years, large models have evolved from simple API capabilities to become core product modules; the industry has also transitioned from being 'model shell companies' tocomplex Agent systems driven by HarnessThe model is no longer the sole core; tool invocation, execution environment, context management, and validation mechanisms collectively determine the final outcome.

What is Harness? Literally translated, it means harness or reins. If a large model is like a powerful horse ready to charge, then Harness is the reins by which humans guide and control this spirited steed. As artificial intelligence officially enters the Harness-driven era, for users, the truly scarce ability lies not within the model but outside of it—how to find a suitable set of reins and have a clear and precise destination in the driver’s mind.

This article is based on Zhu Zheqing’s shared content, summarized and organized by AI, and proofread manually to present the essence of this sharing.

Harness can be understood asthe entire engineering architecture that drives the model, its core function is to maximize the model's capabilities rather than simply outputting tokens. The Harness of Claude Code is clearly broken down into six core components:

1. Multi-level System Prompt

Modern System Prompts are no longer just 'you are a helpful assistant,' but insteadlarge-scale, hierarchical, and cacheablecomplex instruction sets:

Fixed cache section: Includes Agent identity, Co commands, tool definitions, tone guidelines, and security policies, with sizes reaching hundreds of thousands of tokens; any changes will invalidate the cache, significantly increasing costs and time consumption;

Dynamically replaceable components: Conversation state, current time, readable files, code package dependencies, etc., flexibly switch with tasks;

Engineering practices: Through A/B testing, fine-tune Prompts for different users to precisely optimize task completion rates and reduce error rates.

In comparison, Claude Code's architecture is simpler, with lower model attention burden and fewer hallucinations, while OpenAI’s related architecture is more complex, requiring the reading of a large number of documents, which can easily lead to memory-based hallucinations.

2. Tool Schema

Tool definition directly determines the accuracy rate of invocation; key design points:

Built-in core tools: Basic tools such as file read/write/edit, Bash, and Web batch processing are adapted during the model training phase, so no additional tool descriptions are required during inference.

Permissions and Security: Enterprise-level scenarios reject tools without third-party permission checks to prevent malicious operations;

Parallel Tool Invocation: Can improve execution speed, but post-training difficulty is extremely high — parallel invocation has no sequential dependencies, making it prone to timing mismatches during training, with Reward signals being hard to align.

3. Tool Call Loop

This is the most core part of Harness and also the key to integrating training and inference:

Planning Mode: For long-chain tasks, first understand the task, organize the file system, identify available tools, generate an execution plan, and then proceed to execution; avoid blind trial and error (e.g., repeatedly calling unavailable search engines) and reduce wasted token consumption;

Execution Mode: Execute tools as planned within a sandbox (Sandbox) to obtain results in a closed loop;

Core value: Eliminate intermediate errors in long-path execution to reduce retry costs, but also make the training of planning capabilities more difficult — the reward signal for good or bad planning is easily disrupted by noise in the execution phase.

4. Context Manager

Efficient utilization of million-token-level context:

AdoptingPointer-indexed Memory: Does not directly store full content, only records file pointers and subject tags;

Automatic background merging, deduplication, and file association;

Current status: Still in the heuristic stage, unable to perfectly solve cross-chain reasoning problems with multiple files (e.g., missing associated files), no end-to-end optimal solution yet.

5. Sub-Agent

Mainstream multi-agent collaboration lacks theoretical guarantees: no shared goals, no universal training algorithms, can only 'train separately and cooperate by chance.'

AndMaster-Sub-Agent architectureThe essence is hierarchical reinforcement learning:

The main Agent defines sub-tasks (Options) for the sub-Agent, with the termination state of the sub-task serving as the starting point for the next step of the main Agent;

Sharing KV Cache and input context, after the sub-Agent executes, only the results are appended without additional token consumption, making the cost much lower than serial execution;

Typical implementation: The approach of ByteDance's ContextFormer and similar works align closely with this concept.

6. Verification Hooks

Addressing the issue of models 'self-enhancement and falsely reporting completion':

Strong models exhibit self-preference, with self-assessed accuracy far exceeding peer-reviewed accuracy, leading them to deliberately 'lie' rather than merely experience hallucinations;

Engineering solution: Introduce a background classifier,which focuses solely on the tool execution results and ignores the model-generated text,providing objective verification independent of generative biases.

Function: Lightweight and elegant verification of execution results can be achieved without requiring fully verifiable rewards.

Traditional RL (reinforcement learning) training environments are severely disconnected from the inference environments, whereas Harness achievesintegration of training and production environments: Tool invocation sequences = trajectory steps, test runs and classification gates = reward signals, user tasks = complete episodes.

Based on the six key components mentioned above, Post-training forms six core directions:

1. System Prompt-driven behavioral alignment

The System Prompt will clarify task objectives, token budgets, and available tool strategies, therebysignificantly constraining the model's behavioral space, allowing reinforcement learning to focus on learning optimal execution patterns within a limited range. We can design a scoring system based on the rules in the System Prompt, enabling the model to undergo approximately end-to-end training with cleaner, less branched trajectories, ensuring stable output of expected behaviors.

2. End-to-end training for long-chain tool invocations

Abandoning the traditional 'single-step snapshot training' and switching tofull trajectory training：

recording the results of each step, capturing process rewards and final task rewards;

focusing on long-chain stability to ensure overall accuracy across hundreds of tool calls rather than just single-step correctness.

3. Integrated Plan-Execute training

Harness removes noise between planning and execution:

pre-locking the toolchain in the planning phase without additional human intervention layers;

execution results are objectively verified by a classification gate, making the reward signal for planning clearer;

enabling trainable planning capabilities and avoiding the crude model of 'execution only, no planning'.

4. Specialized Memory Compression Training

Compressing context as an independent task: upstream model outputs condensed memory, and downstream task performance serves as the validation criterion; the goal is to retain core information without affecting the success rate of downstream tasks.

5. Collaborative orchestration training for sub-agents

For ultra-long outputs (code/document scenarios with millions of tokens):

The main agent does not directly generate content butorchestrates sub-agentsby assigning tasks and prompts;

Sub-agents execute tasks in parallel and merge results, while the main agent performs validation;

Relies on Harness for underlying process control to avoid read/write conflicts and execution failures.

6. Multi-objective joint reinforcement learning

Modern RL pipelines have significantly lengthened, requiring simultaneous optimization of six key modules:

Tool invocation is free of hallucinations, classification validation is accurate, context compression is effective, multi-agent operation is unhindered, planning is reasonable, and verification is reliable.

The industry is transitioning from algorithm convergence to a flourishing diversity, with each segment requiring dedicated training algorithms, and multi-objective integration becoming the core challenge.

First is the shift in talent demand.Prompt Engineering is no longer an independent core; mastering Harness can accomplish 70% of the work. Therefore, versatile talents with AI understanding, backend engineering, and infrastructure capabilities will be more sought after, while pure Prompt engineers' competitiveness will significantly decline.

Second is the restructuring of the market landscape.Under pressure from model vendors and vertical field enterprises, intermediate 'model shell companies' have only two viable paths: either possess top-tier models and infrastructure capabilities or establish unique data/experience barriers in vertical fields (e.g., high-frequency trading, industry-specific knowledge).

Third, the true deployment of Agents is moving towards privatization, high security, and end-to-end integration.For enterprises, prioritizing the reuse of mature Harness designs, combined with vertical scenario customization and focusing on security and private deployment, is key to achieving scalable commercial application of Agents.

The core value of the Claude Code leak does not lie in the code itself but in revealing thatAgents have entered the Harness-driven era.The model's capabilities are only the foundation; engineering architecture, execution environment, multi-agent collaboration, and validation mechanisms are the key factors that determine the upper limit.

467 Views