Understanding Frontier Artificial Intelligence

kennywinker

Color me skeptical. LLMs seem to make writing code faster, so of course that means that people can iterate on ideas faster, but I have yet to see actual creative output from an LLM that wasn't coached into it or random juxtaposition.

sgt101

This is much better: https://www.rameznaam.com/p/471bbae4-1163-4048-944b-18f8b0bf...

lordnacho

At what point is human intelligence going to hold back machine intelligence? Imagine you are evaluating what the machine should do when it is improving itself. It does a bunch of work and returns with "I supervaluated the liminal overdecomposition from the previous homological calibulation pass. It shows us that subtransitory mulutination will underspecify the tensor of stermullification. Where do you want to go from here?" It will be like when you are reading a Wikipedia about a topic you don't understand. You follow the links, and you get more questions with more links. Your whole day is taken up following links, to the point where you forgot the original question. Except this time, all the words come from the AI's work. You can't refer to an external authority who has already been there and can tell you what to do. The AI needs you to tell it whether it is more intelligent than it was before, but you don't know, because you can't follow its reasoning any more. It's like an ordinary person trying to hire a math professor, there's just no way to do it. But whereas a human math prof can evaluate another one, a machine intelligence can't evaluate another one, by construction. Because it's still usefulness to humans that is the evaluation criterion.

andy_ppp

So predicting the next word given all humanity’s knowledge is surely going to max out at slightly less good (we probably can’t get perfect data) than the best human in any specific field. What test does the AI do to be able to understand it is improving? At some point it becomes impossible to know that the output is actually better right?

sinuhe69

What if automating AI R&D triggers an intelligence explosion? No title editing please

7777777phil

Just thought of this piece [1] by Ramez Naam: Where’s the “intelligence explosion”? [1] https://www.noahpinion.blog/p/wheres-the-intelligence-explos...

jonplackett

I don’t really understand how anyone believes we can control ai that becomes cleverer than us. It’s like 10 monkeys trying to keep a human being a prisoner. It’s just not going to work is it. The entire meaning of being more intelligent is being able to outwit and out-think people less intelligent.

BatchJob

LLMs do not qualify as "frontier AI". They dont even qualify as AI.

bob1029

I think the most promising recursive bootstrapping thing is using the current linear algebra blackboxes to find better ways to construct competitive symbolic models. The ultimate representation for an AI model is an ordinary computer program. Ideally, as a linear tape of instructions. Once we have that kind of a model at the frontier, I think the RSI monster becomes much more plausible.

visarga

I think the premise of runaway intelligence explosion is a kind of naive platonism. It completely ignores the process - how we interact and acquire feedback and validation from outside, and treats intelligence as something that can be ported across domains. My take is that you can only ideate with AI (and brains) but knowledge comes from the contact of those ideas with the world. Making AI better does not make feedback cheaper, faster or more plentiful, it is domain specific. And intelligence does not carry from one domain to another - I might be a good heart surgeon, that does not make me a good investor or AI researcher. Einstein was forgetful, Ramanujan and Godel could not manage simple things like diet. Godel's fear of being poisoned made eating dependent on Adele tasting his food. We all know someone could be a genius in some domain and below average in many other domains. Why does intelligence not simply apply across all domains? Why are our PhD's hyper specialized to their domains and not generalists? Why can't a brilliant scientist simply cure their own dyslexia and still struggle - if intelligence was portable to any domain? The explosion story needs intelligence to be one substance that gets bigger and flows into any domain, I deny intelligence is general.

tim333

The actual title is >What if automating AI R&D triggers an intelligence explosion? It seems to me AI has progressed at a rather steady pace somewhat limited by the hardware slowly improving in a Moore's law like way. Turing in 1950 said it might be 50 years before his test was passed and it seems to have taken more like 75 so not that quick. Even with automating R&D it'll still probably be limited by the compute and so continue steadyish progress. I think we have a bit of a speed up just now due to wild bubble investment and a coming slow down when it crashes but things will plod on overall. That said there may be big society changes because a bit dimmer than humans is very different economically to a bit smarter.