Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5

davidguetta

Don't bet against Google

prodigycorp

Spare me the celebration. Artificial Analysis should not be granting any company the favor of a #1 ranking for a model that isn't released to the public. Google's model release is pure PR puffery. It's no coincidence that it was released on the day of its earnings call.

juujian

Good for them, having a frontier model that nobody can actually use.

isqueiros

I'll use this opportunity while it's still early to share my experience with Argon. I got it via the "Pro" plan, exclusively for personal task usage. So no agentic programming. For starters, the limits are dreadfully low. Five messages on medium is already enough to trigger a 4 hour cooldown. Secondly, the best part is the integration with Google search and Maps. Makes it seamless for trip planning. Lastly it's the only "SOTA" model in a long time that has actively hallucinated details. I don't know if it's because the Gemini harness is not asking it to verify information, but I've had some real Deepseek V3-like lies in responses. Either way, I find it difficult to get invested in Gemini because nobody will have access to Carbon for a few months anyways.

avereveard

Is this a pro 3.1 situation that by the time ot reached the users it produced no more work than the flash variant or competitor for absurd pricing?

miroljub

A secret model I trained last afternoon is the actual #1 on benchmarks. No, you can't use it. You can't even see it, and I won't provide you any information. But you must trust me that I am the genius who made the best model in the whole world.

egeozcan

They can release even Jambon and Jargon, why should I care when I cannot test it?

pranshuchittora

Nice coverup for the Q3 results. Wall St. don't give a F*

iperov

I couldn't care less, honestly

vintermann

I tried Argon for a large historical HTR transcription project recently, since I heard it was really good at this. It was really good, but like all top US models I've used it was * Definitively very good at domain specific knowledge. It can read sloppy 19th century danish gothic handwriting with faded ink just fine. Not perfectly, but probably better than I could have done with a day of struggle for every page. * Expensive. * Bad at actually doing what I said . I had told it in the prompt to be careful not to miss text, but I hadn't explicitly told it that some images may have text in double-column layout. So on those images - over half of my 300-page document - it produced an excellent transcription of the first column, and then considered itself done. Want to try again with a more careful prompt? That'll be another $20.

m00dy

I have pro account and previous model "Argon" still not available on my account.

gavinray

Gemini, for whatever reason, has consistently been the best for explaining scientific concepts for me. Especially with the recent addition of graphic visualization widgets It manages to strike the perfect balance between depth and style of prose that other Frontier models just can't quite seem to match

jonathanstrange

It's unobtanium so nobody should care about it. Any frontier AI company can nowadays create models with ridiculous resource and token usage that will get extremely high scores, but they're of no use when you can't use them. I take this opportunity to give feedback on Argon. The limitations are a problem. I have a Pro subscription and Argon already stops working after 3-4 prompts with long delays afterwards. It's virtually useless right now even when you pay monthly for it. It is better for some tasks than 3.1 Pro, though. Not much in my experience but it can sometimes fix errors that 3.1 has problems with. I never use Flash models since they often give abysmal results. So I'm stuck with using Gemini Pro 3.1 for now.

zarzavat

I don’t care and in my opinion AA does a disservice to their users by allowing such blatant manipulation. What would happen if every AI company were to include their unreleased models like this? Put the unreleased models on a separate page and leave the headline benchmark rankings for models that you can actually use.

quibono

How can one access Carbon? I only see Argon via gemini.google.com Is this API-only or US-only for now?

pcwelder

Gemini 2.5 flash and 3 flash were highly underrated models that performed way better than the peers and we used the hell out of them. Since then none of the Google models have competed well with the others at their price points. Long context performance drop, instruction following issues, unnecessarily complex and sloppish outputs. Argon is no different, it feels just an iteration better than 3.8 flash. I don't trust the AA ranking yet.

m00dy

well, $GOOG is down 13%...

yipinwong

Doesn't mean jack without access to it. I used to defend Google's models months ago, but switched to Luna for less intelligent models. But with this bs of not making it even public, pushing off release of newer models late, I lost faith in Google.

parsimo2010

Just so everyone is aware of the "coincidences" that happen with this index: - Two days ago AA announced their v5 Index which added a few new benchmarks and tweaked a few things. - Today the newest model announced from a big lab tops the charts. I'm not saying that AA is definitely taking money from the big labs and changing the benchmark to make the labs' new models look better, but it kind of looks that way. This pattern of behavior has happened before, where the index gets tweaked right around when a new model comes out, and the new model is on top. AA does not, AFAIK, provide a way to see how the new models would have scored under the older index versions. You could probably calculate it yourself, but I'm not going to do that much legwork just for an HN comment. It's interesting because they publish the index methodology, so you can't really accuse them of not being transparent, but I don't really believe that the choices to update the index were completely independent of the model they happened to be evaluating at the time. Related: The DXOMark camera scoring method also had similar behavior, where they would release new testing methodology right before flagship smartphones were released. Again, they are technically being transparent about the process, but it kind of smells like the "independent" testers may not be as independent as the average user would hope.