How the Claude Code team does code review

crooked-v

Well, that explains the Github repo with 5000 issues, maybe about a third of which are random regressions of previous behavior.

jstummbillig

That's a shockingly low number.

mixtureoftakes

30 thousand changes merged per day? no. like just no. what are we doing? I'm trying to understand

vavkamil

> Security gets its own automated review too. ... You either fix it or close the thread with a short reason, which is how you clear a false positive. I hope this is only for low-risk false positives. Dismissing a critical finding as a false positive should still involve AppSec. You could improve revert rates while simultaneously introducing more security vulnerabilities. And the linked "claude-code-security-review" explicitly states in the README > This action is not hardened against prompt injection attacks That's a significant limitation for an AI security review control, no?

hyperpape

I'm somewhat surprised they would say Claude is writing 90% of their code at this point. Did they get tired of reporting new milestones, or has there been a plateau?

ChrisArchitect

Title is: How the Claude Code team does code review

dylanhouli

Here I am still unable to stop myself from doing line-by-line code review. I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) by doing it that I feel it would be careless not to.

thisisauserid

Remind me to invest more in the clouds they are burning those tokens in.

gonzalohm

That explains why the UI keeps changing. How is this a proper product? Every time you open the desktop app the buttons may be on a completely random place

JackSlateur

90% ? A random twitter said that, 6 month ago, it was 100% ! DECLINE ! SELL THE STOCK OPTIONS ! (wait, there are none, confused!)

nakul_

I wonder how they maintain the quality of tests for such a massive codebase and frequent commits.

typesanitizer

FWIW, Pangram identifies several sections as 100% AI-written.

stymaar

Poorly I'm afraid, given how many new bugs make their way to the claude code CLI every update.

luisgvv

And yet they can't fix the janky bugs when you scroll or show/hide chain of though, commands executed, configuring plug-ins from the options menu etc.

rileymichael

what are people using for automated code review these days? the claude code review product originally mentioned it cost "$15-25 per review" (which is absurd), but it seems like they've removed that from their docs with the launch yesterday. we've looked at various products in the space (coderabbit, greptile, etc.) but have gripes with each. honestly i'm thinking it might be easier (and cheaper) to launch our own agent from GH webhooks and trigger `/code-review`..

koolba

> Low - Docs, copy, internal-only features, additive changes behind a flag > None required once automated review and security checks are green How anybody can let an LLM write docs and not even read them is pure insanity.

RGS1811

How is the ability of a human to understand the code via hands-on familiarity not a direct dependency of their ability to do manual code review? I don't get it. Having claude write an explainer (something I do a lot!) doesn't get you to the point where you can confidently assess a complex diff. And you've funneled your priors on what the diff says and does through the same system that already reviewed and approved it. This article isn't bad but it doesn't give enough weight to the human context-building part of the problem.

ativzzz

It looks like they are specifically optimizing for incidents/reverts and shipping faster > The goal hasn't changed: less human time per change, and no more reverts than before. Is this Goodhart's law in action? Less reverts and less incidents was measured and became the target. Is this then still a good measure? I guess Revenue is a more direct measure, but we don't have access to that one.

jwpapi

What I’m more worried about is feature identification and how to monitor them. How can you get the product to the next level? Honestly I’m not worried about code anymore in my codebase I’ve not had bug problems for a while and incidents get fixed automatically. I monitor new features with an agent who checks recordings. Most actually good features need still 20-30 iterations to design the userflow to actually be interactive, but then there is also another challenge to translate usage patterns into features and how to present functionality with the least amount of friction as possible. How to efficiently run monitors without wasting your token spends. A best 'monitor' best practice would be nice. Still this article was very helpful and gave me a lot of new ideas.

YmFzZTY0ZGFkZHk

And so the wildly unprofitable AI company explains how it token maxes it's code and review process. Authors should really try and view this article from the perspective of someone who has to think about cost to use the models.