Key points
- Using a single H100 GPU, FlashProve produces proofs at a rate of 110 tokens per second for the 400B-parameter Llama 4 Maverick model.
- The system uses zkSNARKs to prove inference execution without revealing model weights.
- Proof generation takes 70-95 times longer than standard inference using vLLM.
Anthropic researchers have published details on a system called FlashProve that generates cryptographic proofs for large language model inference. The work aims to solve the problem of verifying that a response came from a specific model rather than a substituted one.This new system creates digital receipts that prove an AI response came from the specific model you think it did.FlashProve leverages zkSNARKs to achieve 110 tokens per second proof generation for 400B-parameter models on H100 GPUs.
The evidence
According to a research post by Anthropic, the team made progress on zkSNARKs for inference.reported They built a system substantially faster than prior open-source systems. Using a single H100 GPU, the system produces proofs at a rate of approximately 110 tokens per second for the 400-billion-parameter Llama 4 Maverick model. Proofs are roughly 19 MB for every 1,024-token request with 4-bit quantized weights. This performance was achieved using integer arithmetic in prefill.
The decision
Model providers can use cryptographic proofs of inference to commit to a specific model. This enables a model provider to demonstrate that every output originated from the designated model. These proofs could support uses spanning from consumer goods to internal security measures. These proofs might also help verify adherence to AI pacing regulations.
What the evidence means
The system is based on established design principles for cryptographic proof systems. Claude discovered innovative methods for utilizing cutting-edge resources from cryptography and security research. The model's weights are concealed by the commitment to the model and the resulting inference proofs. The system does not aim to conceal the model's architecture.
What builders can do with it
Developers can use FlashProve to verify that responses originate from a claimed model. This addresses the inability of users to check if a response came from a specific model. Builders can implement this to prevent secret substitution by routing services or attackers. The system allows for verification without exposing model weights.
Why it matters
- It provides a technical method to verify model identity, addressing trust issues in AI responses.
- The speed of 110 tokens per second makes real-time verification potentially feasible for some use cases.
- It offers a path toward verifying compliance with AI pacing requirements through cryptographic means.
- The system hides model weights while proving inference, protecting intellectual property during verification.
Topics: Anthropic, zkSNARK, FlashProve, provable inference, AI security
Updated
For readers
Developers
- Can verify model identity without exposing weights.
- Must account for 70-95x slowdown in proof generation.
Policy
- Could help verify compliance with AI pacing requirements.
- Provides a technical basis for auditing model usage.
Consumers
- May gain assurance that responses come from claimed models.
- Could protect against secret model substitution by services.
Ask the article
Local keyword search of this article; no generated answers.Did this article give you the evidence you needed?
Your choice stays on this device. No public tally.Questions readers ask
How fast is FlashProve compared to running the model itself?
The process requires 70 to 95 times more GPU time than executing the model via vLLM.
Does the system hide the model architecture?
This system does not aim to conceal the model's architecture.
What type of cryptographic proof does FlashProve use?
It uses zkSNARKs, which are noted for strong hiding and efficiency properties.
Connected stories
Reading receipt
- Using a single H100 GPU, FlashProve produces proofs at a rate of 110 tokens per second for the 400B-parameter Llama 4 Maverick model.
- The system uses zkSNARKs to prove inference execution without revealing model weights.
- Proof generation takes 70-95 times longer than standard inference using vLLM.
Still open: How will this system scale for smaller models or different hardware?
Still open: What are the specific costs of implementing this in consumer products?