Which AI model really answered? Anthropic shows a proof anyone can check

TL;DR. If you use an AI through an API, all you see is the answer. Whether the model you pay for is doing the work, or a cheaper one, cannot be checked today. On November 2, Anthropic presented FlashProve, an open research prototype meant to change exactly that: the provider delivers a proof with every answer, and you can check it on an ordinary computer without seeing the model's weights. We built the verifier ourselves. It checked a full-size proof in ten seconds on a MacBook and rejected all 20 tampered versions. The price is high: proving costs the provider 70 to 95 times as much compute as the answer itself.

The problem: all you see is the answer

A model is hard to recognize from its answer. A provider can use a smaller model, compress the weights more heavily or pass requests on through a middleman. At most, the customer notices a drop in quality. That this happens is documented. In summer 2024, researchers from Stanford measured commercial APIs for four Llama models: 11 of 31 endpoints returned answers that did not statistically match Meta's original weights. A group from Berkeley concluded in 2025 that such tests alone remain unreliable. They need many requests and miss subtle changes.

Anthropic names two cases explicitly in its announcement: routing services that secretly substitute another model to save costs, and attackers who alter answers on the way.

What FlashProve does

The method comes from cryptography and is called a zkSNARK: a short proof that reveals nothing about the secrets it is made from. The provider commits once to a model and publishes a fingerprint for it. With every answer, it produces a proof. Anyone who has the fingerprint, their own question and the answer can check whether exactly this model gave exactly this answer. The weights, the model's trade secret, stay hidden.

The numbers from the announcement of November 2: the test model was Meta's open model Llama 4 Maverick with 400 billion parameters. A single H100 GPU produces proofs at about 110 tokens per second. A proof for a request of 1,024 tokens is about 19 MB. The only earlier proof for the same model that Anthropic knows of took 14.3 hours for 1,000 tokens. According to Anthropic, Claude built the system. So that nobody has to trust that blindly, the central guarantees are proven by machine in the Lean proof assistant. Code and proofs are available under an open license on GitHub.

Our test: the verifier on an ordinary computer

Only someone with an H100 and about 290 GB of memory can produce proofs. The check, on the other hand, is meant to run on any computer. We tried that on November 5. The project ships a stored full-size proof for this: a model with the shape of Llama 4 Maverick, but with generated random weights, and an answer of 1,001 tokens. Tokens are the word pieces that models compute in.

• Checking is fast. In three runs the verifier said yes after 9.7 to 10.9 seconds and used up to 3.6 GB of memory. You download a 6.2 MB key once and an 18.8 MB proof per answer.
• Every change is caught. We altered the files 20 times, each time in just one place. The verifier rejected all 20 versions, mostly after three seconds.
• Too little memory does not mean yes. On a small test machine with 4 GB, the large check broke off and reported "not checked". On the intended route with Linux, the small example passed there, and all 79 altered versions from the project's test table failed.

On the Mac we compiled the verifier with Apple's own compiler, which the project does not officially support. And the weights in the example are generated. So the test shows that the check works on an ordinary computer and detects changes. It does not show that a real provider serves a particular model.

The catch

• Expensive. To prove everything one GPU serves would take 93 GPUs, by Anthropic's calculation.

• Short. So far the method covers 1,024 tokens, a short exchange. Today's models handle hundreds of thousands.
• Strict or loose. What is proven is an integer version of the model. Its first choice differs from the usual model at 6.81% of positions. So of 96 normally produced answers each, none passed the strict comparison. For operation, a looser check is planned that says much less: answers in which one token had been swapped for the model's second choice got through in 28 of 32 cases.
• The fingerprint names no model. A provider can commit to any model, including a small one. So the fingerprint must come from a party you trust, never from the server being checked.
• The method is proven, not the program. The fast verifier for everyday use is tested, but not formally proven. What running times could give away is open.
• Not a product. The project calls itself a research artifact and is not maintained. According to TechCrunch, neither Anthropic nor any other lab plans to deploy it. Anthropic secures its own traffic another way: answers are signed and recomputed on a trusted server.

What this means for mid-sized companies

For most applications the question is secondary today, as Anthropic itself writes. It matters in three cases. First, when you pay for a specific, expensive model through a middleman or reseller. Second, for open models that third parties run for you: that is where the Stanford measurement found its deviations. Third, when you later have to show which system prepared a decision, for example to auditors, customers or authorities.

FlashProve shows that such a proof is technically possible and getting cheaper fast. You cannot buy it yet. Until then: secure trust by contract and measure for yourself.

What you should do now

1. Pin down model and version. In the contract or the order: exact model name, version, compression, data center. Changes only with notice.
2. Log the model name. Many APIs name the model in every answer. Store that along with the answer. It is no proof, but the basis for any later clarification.
3. Keep your own test questions. 20 to 50 fixed questions with known good answers, asked once a month. If the result suddenly drops, ask. The Stanford group detected deviations this way in about three out of four cases, with an average of ten answers per question.
4. Ask middlemen. Who actually computes, and in which version? Get it in writing.
5. Ask for evidence in sensitive applications. Today that means signed logs and sealed computing environments whose hardware confirms which program is running. Cryptographic proofs will join them once they are affordable in operation.

With AI services, trust has been the only option so far. FlashProve shows that this can change, even if it will take time. If you want to align your AI contracts and test questions with this, we are happy to help.