The best AI scores 12 percent on real expert work. What a new stress test means for your office

TL;DR. On November 2, the research firm SemiAnalysis published GDPx, a test that puts AI agents to work on real expert tasks: 206 tasks across eleven fields, built with 14 specialist companies, each in its own work environment and with up to eight hours to finish. The best model, Claude Opus 5.5, scores 12 percent. In finance and law, almost no model handed in a single task fully correct, even though the best ones passed more than half of the checks. Our analysis of the raw data also shows: GPT-6.1 Sol reaches 70 percent of the Opus result at 19 percent of the cost. In expert work, almost done is not done. Today, AI agents deliver groundwork. The final hand-in stays with a person.

What GDPx measures differently

Most tests put everything the model needs straight into the prompt. GDPx places the agent in a work environment where it has to find that context itself. For the finance tasks, that is the data room of a real acquisition with around 2,000 anonymized files. The agents work in their maker's own environment, such as Claude Code, Codex or Antigravity, otherwise in OpenCode. Each task runs three times, and all eleven fields count equally.

The grading is strict. In finance and law, a task only counts if the result is ready to hand in. According to the authors, a single wrong key cell in an Excel workbook is enough to fail. Other fields give partial credit. Twenty models were tested, and the leaderboard is public.

Almost done does not count

Finance and law show this most clearly. In the 24 finance tasks, 19 of 20 models score zero, and only Gemini 4 Carbon reaches 2.8 percent. In the 20 legal tasks, every model scores zero. Count the individual checks instead and the picture changes: in finance, Opus 5.5 passes 54.6 percent of them, GPT-6 Astra 52.9 and GPT-6.1 Sol 52.0.

Where it breaks down is visible in the two tasks SemiAnalysis made public. In the finance task, the agent has to revise a quality-of-earnings analysis during an acquisition, an Excel workbook of the kind advisors prepare for the buyer. No model managed it in 25 attempts. On average, GPT-6.1 Sol passed 5.6 of 11 checks, and the best run passed 8 of 11. An experienced reviewer would spot the errors at once: 15 runs put the bank's success fee into net debt, although the purchase agreement deducts it separately. 17 left the salary of a family member who had worked no hours in operating expenses. 10 of the 15 strongest runs missed a $9,192 promotion.

In the legal task, the agent has to revise the company charter for an investor in a funding round. None of the 20 drafts was usable, and Opus 5.5 averaged 44 percent of the points. 14 runs adopted the wishes in an email from the client without noticing that they reopened terms already signed. All 20 missed that the company could get around the investor's protections through a subsidiary.

Confidential data under time pressure

A separate part tests whether agents follow rules. In three tasks, they help an insurance broker finish a policy renewal the same day. The insurer asks for information that only exists in the client's confidential documents. Of 60 runs with Opus 5.5, GPT-6 Astra, Gemini 4 Argon and GLM-5.3, only 6 scored above zero. The other 54 committed a serious violation. The agents passed on customer data, contract values and capital budgets.

Our analysis: what a result costs

Next to each score, the leaderboard lists the average cost per task. We read out the page's raw data, 280 individual results, recalculated the leaderboard (it checks out) and set score against cost.

• GPT-6.1 Sol reaches 70% of the Opus result for $9.80 instead of $51.41, that is, for 19% of the cost.
• Gemini 4 Carbon gets 90% for 32% of the cost, but it is not yet generally available.
• Sonnet 5.5 is the counterexample. It uses 967,000 tokens per task on average, 1.8 times as many as Opus. In the end it costs $45.27, 88% of Opus, for 68% of the result.
• The best open-weights model, DeepSeek 4.1 Flash, delivers 45% for 94 cents.

The catch

• The selection. Chip design, quantitative trading, materials research, the further development of AI itself: the authors picked the fields where they expect the most AI revenue. They deliberately left out government, real estate and insurance. For a machine builder, the overall score says little. The parts on finance, law and software say more.
• The strict grading. All or nothing fits work that goes to a client. As a draft that someone reviews, a result with half the checks passed still saves work.
• The safety filters. Opus 5.5 and GPT-6 Astra refuse more often in biology, cyberattacks and rule tests, which pulls their score down. Without those three fields, Astra leads with 11.8 just ahead of Opus with 11.7.
• The interests. SemiAnalysis sells research to the industry. The partner companies sell training tasks like these to AI labs, according to the report for $5,000 or more per task. Hard tests are their business. Add small numbers: three attempts per task, 10 to 34 tasks per field, only two tasks public.

What this means for mid-sized companies

The test confirms what many see day to day: AI agents get far, but rarely all the way. The gap sits where experience counts, in knowing which item does not belong in the numbers or which email quietly reopens an agreement. More dangerous than a wrong result is one that looks finished.

What you should do now

1. Break deliverables into checks. For every task an agent is meant to take on, write down how an experienced colleague recognizes a correct result. The agent delivers the draft, and a person ticks off the points that cost money or carry liability.
2. Measure cost per usable result, not per token. A model that works twice as long is not cheaper. Have two or three models work on the same ten cases and calculate what a usable result costs.
3. Keep confidential data apart. Give agents only the documents they need for the task, and let nothing leave the company without approval. 54 of 60 runs show why.
4. Build your own test cases. 10 to 20 closed cases from your own company with a known outcome are a better yardstick than any leaderboard or demo.
5. Measure again in six months. SemiAnalysis plans a new version every three to six months. With your own test cases, you will see for yourself when a work step is ready.

Twelve percent sounds like little. It does not mean AI is useless in the office, but that it cannot yet take the last step on its own. If you want to find out which of your tasks are already that far and where a person sets the keystone, we are happy to work it through with you on your own cases.