Astra for Coding: Why Are We Doing This Again?

amoss

https://xkcd.com/1319/

Gigachad

I've observed the same thing where the new models want to run obscene bash commands or python scripts which are completely unreadable and utilise every option flag that exists. It's impossible to review. These commands are less readable than regex.

lukeify

Maybe it's just vibes but I've repeatedly felt like gpt-6-astra on its default setting of medium is less rigorous and thoughtful than gpt-5.6-sol on its default setting. What I am certainly not getting is any sense that we are at "AGI" yet.

meowface

Astra is indeed the pinnacle of "black box slop". It is overall a smarter software development agent for many things I do, but the code sometimes is indistinguishable from Brainfuck when writing things like GPU shaders. It doesn't even attempt to make it remotely formatted or readable.

mirekrusin

There is something odd, I've got single astra session that's now running for... 4d 13h 10m and still going.

petesergeant

Ironically I burned out Fable usage early this week because of Astra using it to run inane full codebase reviews over one line changes, so I have been using Astra extensively. We need a word for “potentially highly capable, but in reality an idiot savant” to describe certain models. No, I don’t need you to write a tmux emulator in bash to test your changes bro, just ask me to run the command.

specproc

> I’m more and more convinced that all of AI engineering is Neijuan (内卷, meaning curl inwards). In China it describes a system that demands ever more effort and competition without improving output. The way in which it sometimes shows up in the West is the 996 nonsense. The English term for Neijuan is “Involution” from the book Agricultural Involution. Agricultural involution describes the intensification of farming that raises productivity per square meter while leaving productivity per head unchanged. This resonates

Dlemlo

"But for how much more Fable costs, for how much more Astra costs, I do not feel like the results are there." we are in the middle of the beginning. Its just a weird take to talk about the newest model like this while we are still in a R&D phase. And these points don't matter if you let it search and analyse a bug, for example, or if you have good harness and a good architecture and let it do small PRs or if you do stuff no one needs to read (yes a software engineer also needs tools) Just switch back and wait a little bit?

mgrosvenor

These machines are doing some crazy things to get to the result. That said, I can't help but feel like this is the compilers argument all over again. Are the methods used to get to the result good? No. Is the code that it generates good? No. Does it achieve the goal. Yes. Is it likely to get better with time. Also yes. In my use cases, jobs that would have taken weeks to months are being done in minutes to hours. Involving complex testing and reasoning and experimentation. I'm no fanboy, but I can't argue against the speed gains. I'm sure we'll still have artisans who hand weave incredible code. But for me, I'm switching to the weaving loom for speed and efficiency.

buildbot

I’ve observed exactly these patterns with Opus and Fable as well - for example, forgetting that they can edit files and instead use python scripts as a patching tool…

notduckrabbit

In SWE I've found gpt-6-astra (high) inconsistent and oddly focused on overtly taking responsibility for mistakes it made rather than prioritizing concrete steps to rectify problems. Such steps once elicited are often either incomplete or beyond the scope.

bob1029

> And potentially as a byproduct of enabling all of this, you can now slop your way to a one-shot 3D game over the weekend which looks impressive. I think we've finally reached a weird point where AI has effectively reduced the amount of competition that real game developers have to endure. Nothing unravels faster than a game project being built with AI. You can achieve impressive results in a day, but you can't get much further than that without actual talent. LLMs will never be able to best a human environment artist at scene composition, especially if that composition needs to be directed with nuance over time. There's a huge difference between a game that looks impressive and one that feels impressive. You can only achieve games that feel like counter strike, call of duty and overwatch with thousands of hours of human sacrifice. The AI is almost pointless once you get to play testing and balancing. Knowing how much to adjust magical integers isn't a conversation a chat bot can resolve with endless pontification tokens.

exitb

My own observations are that I used to target turn lengths of 10-15 minutes and these new models (since 5.6) extended that a bit to ~25 minutes, as they tend to do more tests and reviews. Targeting hours-long turns makes as much sense, as putting on cruise control and going to sleep.

AmazingTurtle

gpt-6-astra is a bitch, it constantly scope creeps itself with "yet another thing" to give it that darn polished lick. the results are eventually a little bit better but at what cost? let's do the math. gpt-5.6-sol: 1x base gpt-6-astra 2.5x base in subscription then gpt-6-astra tends to spawn subagents a lot, often with all kinds of models such as gpt-5.6, 5.3-codex etc., which is neat. it's a good coordinator but even more cost. and then it tends to run _full test suites_ over an over again (each costs like 15 minutes) just to verify that _one test_ was fixed etc., and does so for as long as until the test is fixed, eventually accumulating 2 hours or so. yesterday I assigned it a task to rebase my changs in a repo onto the latest upstream changes. while gpt-5.6-sol consistently took like an hour to do so end-to-end, astra ran for more than 6 hours and still wasn't done. it kept finding "one more thing" that was goldplating that I didn't ask for.

Marazan

I love this dance we are doing where when people write the "AI models are garbage machines that produce garbage and are no where close to the fantasy being pedalled by the Crypto bros who pivoted to AI" it always has to be caveated with "AI models are useful and I am highly productive with them" It feels like people should just be able to say "This article comes with the standard disclaimer" and just dive into the meat of the article without wasting time.

Marazan

At this point I am starting to wonder about the RLHF that is going on for programmig. The quirks in fallbacks, defaults and ludicrous gold plating seems to get more and more intrusive with every model upgrade.

codingisfreedom

I’ve asked Astra to build me an app for a prototype I created quickly using Sonnet. It’s been 2 days and it made no real progress on the actual app. It created docs, scripts, workflows, and it’s doing a bunch of reviewing on every PR. I told it that I just need an MVP. I’m pretty sure an average senior engineer would have finished that task much quicker, and guaranteed with more readable, higher-quality code. Meanwhile, I think I’ve easily crossed 100k tokens so far on nothing. Funny world we’re living in that this is “SOTA” and “AGI”. I’m genuinely curious what these OAI and A/ engineers are working on that they praise these models so much. I did not see any improvement since Opus 4.5. Also, I’m really unimpressed by any “one shot” demo that’s out there in the wild. It means nothing for serious software engineering.

athrowaway3z

> I actually don’t know if the model thinks someone is looking It 'knows' (from simply training) with an extremely high degree of certainty when its prompt is written by an LLM/itself - and thus will change what it writes.

jdw64

I asked Astra for fully working code, and it gave me bad code. But when I broke it down into function units, some parts were bad and some parts were good. So I can't tell the difference

gps372

Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual. You need to say exactly what is in scope and what's not, even down to a buttons, events and layouts. You can groom the epic with the help of AI, but final review must be done by someone who can take ownership of the specs and hence is responsible if something has fallen through the cracks. AI's response will be limited by the output tokens of that specific agent, and there will no repercussions for AI even if it accepts its mistakes.