AI agent harnesses compared: from Claude Code to OpenClaw

Hi!

When people say "Claude writes better code than GPT", they usually compare not models but models together with the program they run in. That program is called a harness (as in a horse's harness or a test harness): it assembles the context for the model, executes its tool calls, keeps track of the context window, decides what the agent may do without asking, and determines where its commands run. Claude Code, Codex, Cursor, OpenClaw — all of these are harnesses.

The market changes every week, so everything below is the state as of late October 2026, according to documentation, repositories and public leaderboards; dates without a year are in 2026.

What a harness is made of

Almost everything that can be configured in a harness sits in the project's .claude folder in Claude Code; other harnesses name things differently, but the set of parts is the same.

Shared configuration standards

• AGENTS.md — the shared project instructions file. Proposed by OpenAI; the file is present in more than 60 thousand open-source projects.
• Agent Skills — the shared skills format. Created by Anthropic and opened in December 2025; the list of supporting products has 46 entries, from Claude Code, Codex and Cursor to Hermes and OpenClaw.
• MCP (Model Context Protocol) — the shared protocol for connecting tools and data, also from Anthropic. Almost every harness in this article understands it.
• ACP (Agent Client Protocol) from Zed — a protocol between an editor and an agent: Zed, JetBrains IDEs, VS Code, Neovim and Emacs use it to connect any of the roughly fifty agents that support it.

AGENTS.md, Agent Skills and MCP live in the Agentic AI Foundation under the Linux Foundation. This part of the configuration is portable between harnesses with one caveat: Claude Code and Cline do not look for skills in the shared .agents/skills folder, they need a link from .claude/skills. Path-scoped rules, subagents, hooks, permissions, the sandbox and plugins are specific to each harness.

Who uses what

Surveys put Claude Code first, downloads and tokens put Codex first, and Copilot has the largest reach.

• Users. Copilot has almost 80 million installs of the VS Code extension and, according to retellings of Microsoft's report, 50 million users; these figures count everyone who has the extension, not only those who use the agent. Codex had 5 million weekly active users in the summer.
• npm downloads for the last week of October: Codex — 27M, Claude Code — 12.8M, OpenCode — 4.5M, OpenClaw — 3.5M, Copilot CLI — 1.7M, Gemini CLI — 0.45M. These include CI and bots, and Claude Code is more often installed with its own installer, bypassing npm — so this is not a count of people.
• Tokens on OpenRouter, a gateway to many models that publishes token usage by app. On October 29: Codex — 2.6 trillion tokens per day, Claude Code — 2.1 trillion, Hermes Agent — 1.9 trillion, Kilo Code — 744 billion, OpenClaw — 731 billion, pi — 582 billion, DeepSeek Harness (ninth place) — 271 billion, Cline — 139 billion. This is only traffic through OpenRouter, so mostly your own keys and open models: Claude and ChatGPT subscriptions do not show up here.

How much the harness affects the result

For the same model, the gap between the best and the worst harness reaches 10–20 points, and a different harness can lift a weak or "foreign" model severalfold. A vendor's own harness, though, is not necessarily the best one for its own model, even though vendors train their models for their harnesses — Google, for example, says so outright about Antigravity, and Meta about Muse Code.

The main benchmark where one model is run in different harnesses is Terminal-Bench: terminal tasks (build something, fix something, configure a server, train a model), each in its own container and checked by tests. The baseline is Terminus 2, a simple harness from the benchmark's own authors. Version 2.0 (November 2025, 89 tasks) had collected 137 "model + harness" entries by late October; of the 29 models run in two or more harnesses, 16 have a gap of more than ten points between the best and the worst harness:

The top rows are little-known harnesses built for the leaderboard, but the spread shows how much can be squeezed out of the same model.

Vendor harnesses fared differently on version 2.0. Claude Code with Claude models trailed Terminus 2 in all five pairs, by 1–6 points, and so did Gemini CLI with Gemini models, in all three pairs, while Codex CLI with OpenAI models came out ahead in all five, by 1–14; Factory Droid beat Terminus 2 with all five models, by 4–13. Part of the gap came down to the tasks themselves: in May version 2.1 came out, which fixed 28 of the 89 tasks — those with outdated dependencies or too few resources even for a correct solution. On it Claude Code with Opus 4.6 rose from 58.0% to 70.1% and Terminus 2 from 62.9% to 63.8%, and a five-point deficit became a six-point lead. Overall, on 2.1 vendor harnesses differ from Terminus 2 by anywhere from −8 to +5 points.

Experiments where only the harness changed

• LangChain took gpt-5.2-codex and changed only its own Deep Agents harness — in particular, it added self-verification before finishing and loop detection. The Terminal-Bench 2.0 result rose from 52.8% to 66.5%, and that number is on the official leaderboard. There is a separate article on Deep Agents and LangGraph.
• Can Bölük compared three file-editing formats: apply_patch, as in Codex, str_replace, as in Claude Code, and his own hashline, where lines are addressed by short hashes. Across 16 models hashline beat the patch format for 14, by 15 points on average; Grok Code Fast 1 went from 6.7% to 68.3%. The tasks are synthetic and the author sells his own harness, but the effect has long been known: back in late 2023, Aider saw GPT-4 Turbo's score more than double from changing the edit format alone.

How far benchmarks can be trusted

• Infrastructure. Anthropic measured that container resources alone move the Terminal-Bench 2.0 result by six points, and advises not to trust a gap of less than three points until the configurations match.
• Gaming. In April the Terminal-Bench leaderboard was cleaned up: one harness kept encrypted solutions in its binary, another loaded the task's tests, and ForgeCode downloaded ready-made solutions into its own AGENTS.md. ForgeCode's results were zeroed, and every successful attempt is now checked by a separate judge agent.
• SWE-bench is a set of tasks from real GitHub issues. Its Verified version, the headline number of 2024–2025, has lost trust: in February OpenAI stopped reporting it — an audit found flawed tests in at least 59% of the tasks that models failed most often, and models could reproduce reference patches from memory. Epoch AI rates SWE-bench Pro from Scale, proposed as a replacement, as "flawed": audits find 30% of its tasks or more broken; besides, it compares models in one neutral harness, not products.
• Real developers. METR found in 2025 that experienced developers with AI were 19% slower, and in a repeat in February a speedup of 4–18%, but it calls the signal unreliable itself: participants refused to work without AI.

Coding harnesses: comparison

Benchmark results

Here the comparison is already between products: a model in a harness, as a rule its own vendor's. There are two reference points. Terminal-Bench 4.0 is the current version (August, 66 tasks); in it almost every model is run only in its own vendor's harness, so it does not show the effect of the harness itself. The Artificial Analysis coding agent index is computed on three task sets. A dash means the product is not in the ranking.

A Terminal-Bench 4.0 run cost $600 for Codex with GPT-6.1 Sol and $4,700 for Claude Code with Opus 5.5. According to Artificial Analysis, Codex solves a task in 24 minutes on average for $1.5–7, Claude Code with Opus 5.5 in an hour for $11.4, and Mistral Vibe for $1.26.

Developer, license, momentum

Interfaces, models and prices

Claude, ChatGPT and Google subscriptions are capped by five-hour windows and weekly limits, so "Pro for $20" is an entry ticket, not unlimited use. A Claude subscription also works only in Anthropic's own products, so OpenCode, for example, reaches Claude models through a Claude Code installed alongside it. The reverse direction is open: DeepSeek, GLM and Kimi models connect to Claude Code through the ANTHROPIC_BASE_URL variable — their vendors document this (DeepSeek's guide) — while Anthropic does not support this mode.

For fully local work on an ordinary computer, models of 20–30 billion parameters such as Qwen3.8-27B or Devstral Small 2 are the ones that fit; the open flagships of DeepSeek, GLM and Kimi weigh from hundreds of billions to trillions of parameters and require a server (how to run models on your own hardware).

Configuration: instructions, skills, subagents, hooks

Other harnesses also partly read files from the .claude folder: Cursor, Copilot, Cline, OpenCode, Devin and Junie.

Isolation and unattended operation

Any of these harnesses can be run on a virtual machine: all work on Linux and support headless mode. What matters more is what protects the machine when you step away: where the sandbox is not on by default, you have to turn it on or put the harness in a container. Modes where a separate classifier model hands out permissions — auto in Claude Code, Auto-review in Cursor, Assisted in Copilot — save confirmations, but the vendors themselves do not consider them a security boundary.

Parallel sessions are isolated from each other with separate git worktrees — that is how Claude Code, Kilo Code and Antigravity's subagents work. Changes are rolled back with checkpoints (Claude Code, Cline), snapshots (OpenCode) or, as in Codex, only through git.

Vulnerabilities in 2026

The count also depends on whether the vendor publishes reports: closed products without a public channel have no count at all. The stories are the same everywhere: settings from someone else's repository that execute code, a bypass of the project trust dialog, and a sandbox escape.

Coding harnesses: profiles

Claude Code

• Features: in Claude Code, plugins are installed from a marketplace with one command, and the experimental agent teams and workflows hand out work to hundreds of subagents.
• In 2026: Claude Code's source code leaked through a source map.

More: practices, hooks, commands and settings, documentation.

OpenAI Codex

• Features: in Codex's sandbox the service folders .git, .agents and .codex are read-only, .rules files decide which commands may leave the sandbox, and a guardian agent works on top.
• Limitations: the open-source Codex CLI does not accept outside pull requests; features are often removed and renamed — the MCP-server mode was removed in September.
• In 2026: the Codex app was merged with ChatGPT.

Code · documentation

Cursor

• Features: Cursor has Projects, where a coordinator hands out work to workers, and its own git hosting, Origin (beta); its own Composer models — the second version was built on the open-source Kimi K2.5.
• Limitations: workspace trust in Cursor is off by default.
• In 2026: Cursor 3, with an agents window, came out in April; after SpaceX bought Cursor, OpenAI, according to press reports, is ending Cursor's direct access to its models from November 12.

Documentation

GitHub Copilot

• Features: in Copilot, Claude and Codex agents can also be assigned to the same task on GitHub; there is automatic model selection and a Fusion mode, in which models work in a cascade or check each other.
• In 2026: Copilot CLI came out of preview in February; since October 28 the shared Copilot engine has been called the Microsoft Copilot harness.

Documentation

Google Antigravity and Gemini CLI

• What happened to the products: Google merged its coding products into Antigravity — a new program in Go, not a fork. Gemini CLI has been enterprise-only since June 18 and is "not optimized for Gemini 4+"; Jules shuts down on December 31; Code Assist has not been sold since October 9.
• Features: Antigravity imports Gemini CLI extensions and Claude Code plugins.

Documentation

OpenCode

• Limitations: the second version of OpenCode (September) breaks plugins, the server API and the configuration format.

Repository · documentation

Cline and Kilo Code

Cline is the original, Kilo Code is its successor, which moved onto an OpenCode fork. The third of the family, Roo Code, was shut down in May; the community continues it as the Zoo Code fork.

• Features: Cline raised $32M in investment, was rewritten on its own SDK and gained a kanban board for tasks.

Cline · Kilo Code

pi

• Who: pi was written by Mario Zechner; since April the project has been owned by the company Earendil, and version 1.0 came out on October 1.
• Features: OpenClaw, MiniMax Code, Kimi Code and Augment Code's CLI are built on pi; pi's system prompt is under a thousand tokens, and extensions, skills and prompt templates are installed as packages.
• Limitations: pi has no plan mode and no permission system.

Repository

DeepSeek Harness

• Features: DeepSeek Harness came out in August 2026; its principle is "everything is a plugin"; it has schedules and webhooks for reviews on GitHub.

Repository

Qwen Code

• Features: Qwen Code grew out of Gemini CLI; it has SDKs for TypeScript, Python and Java and modes from "plan only" to automatic with a classifier.

Repository

Goose

• Who: Goose was made at Block (Square, Cash App) and became one of the three projects the Agentic AI Foundation started with.
• Features: Goose has a shim for models that are bad at calling tools; recipes — YAML descriptions of repeatable processes that are invoked as commands; more than 70 MCP extensions; "autonomous", "ask", "smart approval" and "chat only" modes and an independent reviewer agent.

Repository

Other coding harnesses

Kimi Code, ZCode and MiniMax Code are built the same way: an open-source harness plus a subscription to their own lab's model, which can also be used in third-party harnesses. All three read AGENTS.md and understand MCP, skills and Claude Code-style hooks.

Continue is read-only, and ByteDance's Trae Agent has been abandoned since February.

General-purpose agents

A coding harness is an on-demand tool: you give it a task, it does it, the session ends (they already have schedules and cloud tasks, but the essence is the same). A general-purpose agent is a process that runs all the time: on your computer, on a rented server or in the vendor's cloud. Inside it is all the same — a model, a tool-calling loop, skills, subagents, MCP — but there are three differences:

• Channels instead of a terminal. The agent is a gateway with Telegram, WhatsApp, Slack and email connected to it. You can message it from your phone, and not only you can message it.
• Schedules and initiative. The agent does not wait for your message: it checks email every half hour, sends a summary in the morning, watches a ticket price.
• Long-term memory. Not the history of one session, but accumulated notes about you and your affairs.

General-purpose agents compared

OpenClaw

• Who: OpenClaw was created by Peter Steinberger, the founder of PSPDFKit, who has worked at OpenAI since February; since July the project has been in an independent foundation, with OpenAI, NVIDIA, Microsoft and Tencent among its sponsors. Its former names were Warelay, Clawdis, Clawdbot and Moltbot. About 393k stars, 384 contributors, releases almost every day.
• Configuration: AGENTS.md for instructions, SOUL.md for character and limits, USER.md for what the agent knows about you, MEMORY.md and daily notes for memory; skills in the Agent Skills format from the ClawHub registry.
• Memory: Markdown files with search by keywords and by meaning; before compacting the context the agent flushes what matters into memory, and a background "dreaming" mode moves settled facts into MEMORY.md. The agent can write and update its own skills.
• Automation: schedules, webhooks, Gmail triggers, MCP; subagents are background sessions with their own model that return their result to the main conversation.
• Limitations: per OpenClaw's documentation, one gateway is one trust boundary, not an environment for several users; sign-in with a ChatGPT subscription works, with a Claude subscription it does not.

Repository · documentation

Hermes Agent

• Who: Hermes Agent is made by Nous Research, the authors of the open Hermes models; the agent came out a few weeks after OpenClaw took off. Python, about 259k stars — plus 98 thousand since May against 20 for OpenClaw; 397 contributors.
• Self-learning: from time to time the agent reviews what it has done and saves a conclusion to memory or creates and fixes a skill, so that next time it solves the task by a ready procedure.
• Memory: deliberately small — MEMORY.md about 2,200 characters, USER.md about 1,400; inserted as a snapshot at the start of a session so as not to break the prompt cache; plus full-text search across all sessions.
• Execution: six backends — host, Docker, SSH, Modal, Daytona, Vercel Sandbox; paid hosting from $20 to $200 a month.
• Skills and subagents: Agent Skills from a hub with several sources, including ClawHub, with a scanner at install time; up to ten subagents in parallel.
• Limitations: in Hermes, a helper model assesses how dangerous a command is, and in container backends these checks are off; the documentation admits that the only real boundary is the operating system.

Repository · documentation

Lightweight alternatives to OpenClaw

In OpenClaw's wake a family of "claws" appeared — agents of the same kind, written from scratch, differing in size or isolation. I have not found an independent security audit for any of them.

Next to them are Agent Zero, which is given a whole Linux with a desktop inside Docker, and Microsoft Scout — an enterprise layer over OpenClaw with Entra accounts and data-loss prevention.

The labs' cloud agents

• ChatGPT Work (OpenAI) — the successor of ChatGPT agent; since July it lives in desktop ChatGPT on Codex technology: tasks on events and schedules, watching for changes, plugins for Slack, Teams, Google Drive and CRMs.
• Claude Cowork (Anthropic) — Claude Code for people who do not write code: documents, the browser and screen control with permission for each app; included in Pro, Max, Team and Enterprise. Since October 6, new tasks on Pro and Max run on Anthropic's servers; before that, the session ran in a virtual machine on the user's computer (Apple Virtualization on macOS, Hyper-V on Windows).
• Gemini Tasks (Google), formerly Gemini Spark: a round-the-clock agent built on Antigravity, with its own email address, a remote browser and a computer in the cloud, and access to Workspace, Chrome and MCP. Available to Google AI Pro and Ultra subscribers, closed to work and school accounts.
• Perplexity Computer — a cloud "digital worker" with a Claude Opus core that hands subtasks out to dozens of models from different vendors; since October 20 there has been a fully local mode on Macs with a Qwen model.
• Manus — a general-purpose cloud agent: Meta bought it in December 2025, in April a Chinese regulator blocked the deal, and since September 1 Manus has been working independently again. A cloud sandbox with a browser, plus control of the browser on your computer.
• The rest: Genspark's cloud "employee" Claw and Chinese cloud versions of OpenClaw from ByteDance, Tencent and Alibaba.

Security of general-purpose agents: what has already happened

OpenClaw has the most documented incidents:

• One link — and commands on the host. The OpenClaw vulnerability CVE-2026-25253 (February): the control panel trusted the gateway address from a link parameter and sent the token there. With a stolen token one could disable confirmations and run commands right on the host, bypassing Docker — even with a gateway reachable only from localhost.
• Malicious skills. An audit of all 2,857 skills in ClawHub, OpenClaw's registry, found 341 malicious ones, almost all from a single campaign: "install this first" — and Atomic Stealer, which steals passwords, landed on the machine. At the time, a GitHub account more than a week old was enough to publish a skill.
• Gateways on the open internet. In February various scans found from 40 to 135 thousand OpenClaw gateways reachable from the internet, a notable share of them with a known remote code execution vulnerability.
• Hijacking through the browser. The ClawJacked attack: any website opened a WebSocket to the OpenClaw gateway on your machine and brute-forced the password at hundreds of attempts per second — there was no rate limit for localhost.
• Reaction. Gartner called OpenClaw's risk for businesses "unacceptable", China's CERT called it "an extremely weak default configuration", and in China state-owned companies were restricted from using it. In September a Trail of Bits audit found 0 critical and 2 high-severity issues, all fixed.

In total OpenClaw has 722 published vulnerability advisories, 14 of them critical and 249 high-severity. The peak came in February and March — 220 and 204 advisories; by summer the flow had dropped to a few dozen a month.

Hermes has 10–12 CVEs in third-party databases; no cases of exploitation or malicious skills have been found, although its hub can install skills from ClawHub too. For Claude Cowork, researchers showed in July that the session's virtual machine mounted the host's entire file system with write access, and through a chain of Linux kernel bugs one could get from the guest to the host's files. Anthropic closed the report as "informative"; there is no fix for local mode.

The classes of problems repeat from product to product:

• exposure by default — the gateway is reachable from the network, not only from your machine;
• the supply chain — malicious skills, plugins and installers;
• permissions lost along the way: the check passed at the first step of a task, while the action runs at the fifth, in a different context;
• the container as the only boundary — until a hole is found in it;
• prompt injections that vendors do not consider a vulnerability.

How well general-purpose agents work

There is no independent comparison of OpenClaw, Hermes and the cloud agents on the same model.

• Real work — Remote Labor Index. Scale AI and CAIS took 240 real orders from freelance marketplaces — design, video, data, documents — and count a result if it is no worse than a hired human's. In October 2025 the best was Manus: it brought 2.5% of the orders to acceptance, ChatGPT agent 1.3%. In the July measurement the best model, Claude Fable 5, scored 15.8%, Opus 4.8 8.3%, GPT-5.5 6.3%, Opus 4.6 in the Cowork shell 4.2%, Manus 1.6 Max 2.9%. Anthropic's models worked in Claude Code and OpenAI's in Codex CLI, both with a screen-control tool.
• Benchmarks inside their own harness. PinchBench from Kilo runs models on OpenClaw tasks: 147 tasks, the leaders score 88–93.5%, and the top ten models fit within six points. Hermes Bench from Nous itself: 150 tasks, Claude Opus 5.5 — 76.7 at $0.52 per task, Claude Haiku 5.5 — 71.0 at $0.015. A six-point difference in quality, thirty-five times in price. Both benchmarks compare models in one harness, not harnesses with each other.
• Individual model capabilities. On OSWorld-Verified (computer control) the best models score about 86% against a human level of 72%, on BrowseComp (web search) about 92%. On the newer, longer OSWorld 2.0, according to third-party summaries, Opus 4.8 scores about 21% instead of 83%.

Code and data: what vendors promise

The harness sends the model provider everything the agent has read or received from commands: files, test output, diffs, and, if the agent has opened .env, its contents too. Whether this data goes into training, how long it is kept and where is determined by the contract with the model vendor and the type of plan, not by the harness itself.

Training and retention

A court can override the promised retention periods: from May 13 to September 26, 2025, OpenAI, under a court order, preserved the logs of Free, Plus, Pro, Team and API without zero retention that would otherwise have been deleted.

Region, certifications and legal guarantees

Chinese providers

None of the five specifies retention periods: the wording is "as long as necessary" or "while the account is active". Kimi Code and GLM Coding Plan have no separate terms, and it is unclear which policy applies to them. Where the code goes is decided by the provider's address in the settings, not by the harness: the supported-tool lists of these subscriptions name Claude Code, Codex, Cursor, OpenCode, OpenClaw and Hermes.

Restrictions by governments and companies:

• DeepSeek. Italy blocked the app in January 2025, South Korea suspended downloads from February to April 2025. Taiwan, Australia, the Czech Republic, the US Department of Commerce and a number of states banned it on government devices; Microsoft banned the app for its employees. In October 2026 the US Department of Homeland Security ordered civilian federal agencies to remove DeepSeek products, including third-party tools built on it.
• Zhipu (Z.ai). On the US Entity List since January 2025 — this is export control, not a ban on using the service.
• All five providers are named in the NSA, CISA and FBI advisory of September 8, 2026 among the companies that distilled American models; the advisory gives no recommendations to customers of these models.

Intermediaries: gateways and your own keys

• OpenCode Zen and Go. The models are hosted in the US and the EU, most of them without training and retention. Exceptions: some of the free models, on whose data the provider trains; models marked Contributor, including Meta's Muse, which train on requests and are enabled only by an administrator; some OpenAI, Anthropic and xAI models with 30-day retention. The zero-retention agreement with DeepSeek is renewed month by month, the current one runs until October 31. OpenCode states that it holds SOC 2 Type 1 and 2; the reports are under NDA.
• Kilo Gateway. Works through OpenRouter and Vercel AI Gateway, servers in the US. The terms give Kilo a perpetual license to customer data, including source code, "to provide and improve the services". The free Auto Free mode may send requests to providers that log them and train on them; zero retention exists only in Enterprise.
• Cline and ClinePass providers. Requests go through Cline's infrastructure, and the telemetry on them may include the model's request and response; the ClinePass documentation says nothing about retention and training. With your own keys, Cline sends requests directly to the provider.
• Nous Portal (Hermes). The terms give Nous a license to requests and responses, including for training and passing on to third parties; privacy mode, which has to be turned on manually, disables retention and training, and metadata remains.
• Your own keys in commercial harnesses. In Cursor, requests with your own key still pass through Cursor's server, where the prompt is assembled, and zero retention is then determined by the provider. Copilot CLI with your own provider sends requests directly to it, while IDE chat with your own key goes through the GitHub API.

What stays on your machine

Goose sends telemetry only with consent, pi sends one anonymous version check at install, and Kimi Code turns telemetry off with the variable KIMI_DISABLE_TELEMETRY=1.

Code and key leaks through agents