Skip to article Ruben Garcia Jr

Gemini 4 Carbon Is the New #1 — Too Bad You Can’t Use It

All notes ↗

Google announced Gemini 4 Carbon on October 28, and by the one independent scorecard that matters, it’s the best model on the planet right now. Artificial Analysis puts Carbon at 59 on its Intelligence Index v5, first overall, with the top score on four of the index’s nine evals, including HLE-Diamond and Terminal-Bench-Science. On AA-Omniscience it scored 54, six points clear of Claude Opus 5.5 and eight clear of Gemini 4 Argon, Google’s own model from two weeks ago.

Here’s the catch. You can’t run it. I can’t run it. There’s no API model ID and no launch date for the rest of us. The rollout starts with “a subset of cyber defenders” this week, then widens to “trusted partners” through a program Google calls Fairwind. Developers, businesses, and consumers get it “soon.”

The rollout is the story

A number one model nobody can access is a strange kind of victory lap. Google says it’s “actively engaged in the U.S. government’s voluntary process for pre-release model testing” and wants to iterate on guardrails before Carbon reaches its product surfaces. I respect the restraint. I also note that the model Carbon just edged out is already out in the world, earning money for Anthropic while Google polishes a leaderboard it won’t let anyone climb.

What Carbon does have is a very specific job description. The entire launch is aimed at cybersecurity.

A model that finds, exploits, and patches

Google’s benchmark numbers, which the company says were evaluated without guardrails:

  • 88.2% on SEC-bench Pro, autonomously finding real vulnerabilities buried in production software
  • 54.8% on ExploitGym, writing working exploits to prove those flaws are real
  • 71.0% on CWE-bench, turning validated findings into root-cause patches

Find the bug, prove the bug, fix the bug. Vendors grade their own homework, so hold the confetti, but that’s the full defense lifecycle in one model and the gaps between it and the field aren’t small. Google’s internal teams report more than 2x uplift over Argon in generating secure code, and the Chrome team claims Carbon produced patches 10X better than “another lab’s large frontier model.” The other lab goes unnamed.

The part I care about more is long-horizon persistence. Google says Carbon has run for days on multi-step engineering problems, grinding through dead ends to root-cause subtle bugs across large codebases and low-level systems. Anyone who runs agents daily knows the wall I’m describing: hour three, task forty, and the model starts phoning it in. A model that doesn’t do that is worth more than any single benchmark score.

Smarter and less reliable at the same time

One number Google didn’t headline: Artificial Analysis measured Carbon’s hallucination rate at 11%, up from Argon’s 4%. Accuracy climbed ten points to 58% on AA-Omniscience, so the model knows more and fabricates more at the same time. Google’s own post says its internal live-coding environments showed “reduced misalignment and fabrication rates.” Both things can be true, since these are different tests under different conditions. But if you pick models for work where wrong answers cost money, 11% is the number I’d tape to the monitor.

My rule from September still stands: rank models by cost per finished, correct job, not by index score. A genius model that hallucinates gets expensive the moment you stop checking its work.

Two flagships in two weeks

Argon landed October 13. Carbon landed October 28. That’s a two-week frontier cadence at a lab that also found time to gate its best model behind a defender program. Intro pricing matches Argon: $2 per million input tokens, $10 per million output, with cached input 95% off. It doubles to $4 and $20 after the intro period.

The model race used to be decided by who shipped the biggest benchmark. Now half of it is theater. Anthropic is off filing for Nasdaq, OpenAI just cut its training runs off from the live internet, and Google’s play for the top spot is a model most of its customers can’t log into yet. When Carbon actually reaches an API, I’ll grade it the only way that matters to me: cost per finished job. Until then it’s a press release with a leaderboard screenshot.