The Eternal Sloptember

tptacek

AFL didn't find more vulnerabilities than LLMs. AFL and skilled practitioners found vulnerabilities. AFL triggers faults, many (most?) of which aren't exploitable, and humans (or, now, agents) have to triage and evaluate them. And they did so in a pre-AFL corpus of memory-unsafe software . The heyday of AFL was a decade ago. Every target is harder now.

mountainriver

My guess is the models just continue to get better and better When I got into agentic coding a year or two ago I was sure it was only good at autocomplete. Something happened earlier this year where the models hit a new level of capability. Everyone I know now just does agentic coding, and it’s really amazing. I think we should just try pushing this as far as we can possibly go, it really feels like the acceleration of the human race is upon us.

wyager

> They are a highly sophisticated statistical model designed to mimic the distribution of programming Are we really still doing this?

fontain

We all remember cryptocurrency. Everyone in tech proclaimed fiat was dead, every office buzzed with talk of every possible way that cryptocurrency could be used, billions of dollars flooded in to projects losing money hand over fist. The cynics reacted to the froth with outright rejection of the idea. And today… cryptocurrency exists, it has some use, but it didn’t take over the world, it didn’t kill fiat, it was useful in some areas and worthless in others. AI will be the same. The noisiest proponents will be over exaggerating. The most cynical cynics will be underestimating. The result will be somewhere in the middle. Success will not be predicated on adoption of the technology. We, nerds, are bad at predicting the impact of technology.

intended

If nothing else, Eternal Sloptember is a term that seems obvious once you have it. I can’t believe this is the first time I’m seeing it.

spiderfarmer

Coders underestimate the utility of AI in so many boring day to day tasks. If you freelance, that’s where the money is at, not in creating a startup that fills holes in AI offerings or in creating generic slop while hoping for ad money.

farhanhubble

I'm in the "haven't written any code in a while" boat ATM. I'd love to see examples of issues that are so big that they warrant reverting to manual coding. My main issue has been the inconsistent quality across between model releases and the tendency to insert older APIs or documentation, especially with command line tools. I can understand if the model struggles with a million line monolithic codebase with a decade of cruft but can't think of why it'd be too much of a pain with new codebases.

Nition

With the level of ability that AI is at right now, I've found it useful personally to think of it something like a very good search over existing knowledge. Another step up in searchability in the lineage of reference books, stack overflow, GitHub etc. Programmers are rewriting and reinventing the same techniques more often than any other vocation I can think of, and so we were primed for a really good search over prior art. The fact that AI can also adapt that prior art to your particular use case makes it even more powerful. Much like how great success never came from cobbling together various bits of copy-pasted code from Stack Overflow though, current AI can't really build your whole project.

simianwords

Nah this person is dead wrong. Lets come back in 2 years and check on it. I'm willing to make a reasonable bet on these terms: companies will go even more AI native, will use even more tokens and spend even more money. EDIT: To people downvoting me, please come up with a reasonable bet and lets try to work it out. EDIT 2: $500 bet paid to your account on whether LLM's are going to still be used productively or not. No one? EDIT 3: Any bet that would express the author's argument in a way that can be disproven in the future

zarzavat

It really feels like a mass psychosis. I'm not an AI sceptic insofar as I fully expect to get replaced by some future AI system. But what we have now isn't it. To use a Geohot-inspired analogy, what we have now is like the Google self-driving car of 2010. It works most of the time, yet sometimes fails in unpredictable ways. So you need a safety driver behind the wheel to constantly watch what it's doing (the code review). A real AI agent would not need a safety driver. We don't have that but many people are basically saying "fuck it, I'm just going to set this car off on its own and see what happens". And sure if you're prototyping it's not dangerous. But for production systems that is dangerous.

bluegatty

" Agents cannot program, and it’s taking longer and longer to realize that they can’t. They are a highly sophisticated statistical model designed to mimic the distribution of programming" In other words - they can program, and probably better than you. I don't like being too critical but this is a really superficial post - as if either 'AI is a Software Engineer - or - It must be Fraud' It's an extremely powerful tool that is very 'pattern oriented' and with guidance can absolutely write great code - and even across modules given the right basis. It's also great at so many other tasks - finding bugs in big code bases, doing migrations etc. It's not going to make very goo architectural decisions for you, and if you're doing anything novel you have to read most of the code ... but that's too be expected.

blobbers

I don't think LeCun is saying they won't be able to program. I think he says we won't hit AGI. Programming does not require AGI; it's a pretty specific skill! -- I think this article is COPE, if I'm being quite honest. I thought of putting cute analogies, like the C programmers saying the Python and Javascript programmers are not "hardcore" enough... but the truth should be obvious to anyone using LLMs effectively. -- Current AI is a much better programmer than 100% of people and when directed by someone in that top 10%, it's a force majeur.

simianwords

People misunderstand how AI is used in coding in normal work environments. New feature requirement comes - maybe you need a new service or some new classes. You need to do some research first. You guide the AI with some prompts and give it some guidance on how to scenario-test it. It makes some classes, test methods. Maybe ~2000 lines and you do a quick verification, check if the overall idea looks okay. Ask it to fix a few design things and then merge it. Its much easier than doing it yourself with all the boilerplate and understanding each esoteric language specific thing. Which library do I use for UDP communication in golang? The agent might have made a good assumption. These kind of things is where it speeds it up.

nilirl

I agree that I can write better code than an agent. But it can write working code much faster than I can. And in a lot of cases, unfortunately, faster beats better.

protocolture

Eh but statistical models are obviously useful, because statistically 99% of your codebase wont involve new idea invention. Tools that write all the boilerplate code used to have names and job titles. I hate how both the for and against case for LLMs are just so bloody terrible at addressing these things.

Chaosvex

> and it’s taking longer and longer to realize that they can’t For something to take "longer and longer" to realise, doesn't they imply that it's been realised at least once before or that there was an expected deadline for the realisation? Okay, that's a nitpick.

sandruso

Not reviewing outputs, which is my main issue, is one-way to subpar experience. No amount of "make it right" will fix that. I hope that professionalism still matters as these new ways of doing things strikes me as unprofessional as f... Yeah, the next macOS will be worse... time to place bet on prediction market

pipeline_peak

The more specific your work is, the more these LLM’s seem to struggle. If your work was previously googling stack overflow, it can be incredibly useful at working through that. Which let’s face it, that’s what a lot of us do.

coolThingsFirst

Geohot's next venture will be writing a book titled "Fear & Trembling".