How to Build an AI Software Factory: Agents That Open, Review, and Merge PRs

notatoad

This seems like a useful write up of the concepts involved here, but I just hate reading Claude’s writing style so much. This is content marketing, at least give it a review pass before you hit publish… if your marketing is unpolished AI slop, I have to assume your product is too.

demibabs

> Generation scales with spend, review does not, and that asymmetry is the whole design problem. I see you guys have a blog post factory as well.

ahskfkwh

Who tf or what tf wrote this. It’s like I’m talking to opus 5 again, terrible writing. I hate the product already

joshheitzman

We've had software factories for decades. They're usually called compilers, linkers, toolchains, etc.

Incipient

I'm using fable and opus about 25%75/% respectively. I have a 3 step process - discuss a topic and create a review, convert the review to a task file and update the design docs, then finally implement the task file in code. Normally around 300-1000 loc implemented for a change, more for a new UI page or whatever. And honestly it's incredibly good. - 50% of the time it'll generate code that I accept wholesale. - 25%: it maybe makes a choice I don't like, or the implementation isn't the approach I liked, so I tweak it. -10%: it needlessly creates semi-duplicate function logic, so it's a lot more code and execution paths become hard to conceptualise -10%: it mirrors business logic it shouldn't, and subtly changes it. Which needs a huge amount of intervention to fix. - 5% it goes completely off the rails. Eg decided that business/application logic should be in the database. Prompt and check this honestly works very well (still waiting on tests with more than a handful of paying customers however!) but a FULLY automated agent pipeline? I'm not convinced. That last 15% of cases will make a degenerate codebase real fast. Also without a CC subscription, paying per token, it would be horrifically expensive for my approach.