Where does next-token prediction leave us?

sigmar

>But lately I’ve been thinking if it is just a class issue? This cohort of people likely have a cushion that softens the concussive blows they are doling out right now. They perhaps have the luxury of a somewhat functioning government and a social safety net that they are witness to in all walks of life. Over half the world does not. Science and technology, I feel, has always had a certain apathy towards the plight the people at the bottom rungs. In the data I've seen, the US and European countries have a more negative view of AI than China and developing countries. Doesn't that fly in the face of the premise here that only people that have economic security support AI? https://www.visualcapitalist.com/survey-how-21-countries-vie... https://www.ipsos.com/en/conflicting-global-perceptions-arou... https://www.mexc.com/news/161986

anematode

> The compartmentalisation that must be required by the scientists and engineers to reconcile with the fact that their work being used to bomb and kill people must be crazy. I think about a related question pretty often: What proportion of people working at these companies are "true believers", that their work will be a net benefit for humanity? And for those people (if they are at all numerous), how do they plan to fight back against the obvious harms that are already occurring? I just can't imagine working at one of these companies without hating myself. But I suppose with what they're being paid, they can afford a very good therapist...

BobbyTables2

Author seems ill informed. The D.O.W is not exactly a fan of Anthropic…

mmilunic

> Non-technical middle managers who have not written a line of code in their lives, now feel that the biggest obstacle between them and greatness has lifted. I find it interesting how this is almost the “democratization” you mentioned that AI provides. While AI “democratizes” certain technical ability, in some ways the democratization of things can actually be bad, in that this “democratization” pushes us towards a system in which people are completely fungible, and so lose their individual bargaining capability. By democratizing this ability to the non-technical middle manager, the junior software engineer ends up losing their unique contribution and hence vote. I read a while ago about boycotting AI if you can, and I would love to, but this issue makes me wonder if that could even be effective. If the goal is to remove every unique contribution you provide, what can you take away with a boycott?

mlmonkey

I tuned out when I came across this: > Science and technology, I feel, has always had a certain apathy towards the plight [of] the people at the bottom rungs. Does that apply to medical advances too? e.g. antibiotics, vaccines, etc. too? We are living longer today thanks to advances in science and technology. Not just the people at the top; but also the peopl at the bottom rungs. Most scientific research does not take into account who the beneficiaries of that research would be.

perching_aix

Every time someone invokes the "playing God" / "the hubris of creation" card, it makes me think they're the proud ones, and this case is none the different. I don't know why the author is so surprised people want freedom from others. We're the original bullshit machines, and with every useful invention, an additional chunk of utter snakeoil is snuffed out. We're not particularly reliable either. In an old post I can't find for example, I remarked how people can apparently do figure out how to document and coach properly, as long as the target is an LLM, not a human. Suddenly the limits and importance of attention, contextualization, clarity, working memory size, etc. are not so elusive and debatable after all. I'm sure I need not to remind the author of the "certainty of steel" quote, as ironic as that'd be for an indeed inherently stochastic system. A parrot though, I'm not so sure. "Not sure" why the author feels compelled to conditionally deny it is absolutely meant to be pejorative either. One does not need to be blind to the mentioned prospective pitfalls either: https://news.ycombinator.com/item?id=48196923

Lerc

Calling this a straw man would be attributing an unnecessarily sturdy construction. Yes there are people making stupid claims on all sides. Attributing phrasing like solved or cooked is as if it is some sort of fanatic specific jargon simply ignores the terminology of different groups of people. I don't use cooked myself, but I am not so ignorant of the younger generation that is see it is just another in the long line of terms like sick, bogus, hip, radical, macaroni, etc. The author plays the trick of flipping the situation from stochastic parrots or next token prediction. Those are "taken as pejorative" whereas cooked and solved "to signal" The flip is done to place the fault on the other party. You could equally uncharitably say that invoking next token prediction or stochastic parrots is signalling, whereas AI skeptics take terms like cooked as pejorative. Specifically on the topic of next token prediction, we are already past that phase. Even then I don't think that a model trained by prediction has the limitations that people think it does. As a thought terminating cliche it is simply obsolete when models are trained on reinforcement techniques where there is no template next word to predict. Diffusion models don't even have an autoregressive nature. I am not terribly fond of the claims made by people at the extremes of any particular to issue. We can perhaps try debating the facts of the matter rather than assuming the internal thoughts of people who might disagree with you. I generally do not attribute malice to people who describe models as next word predictors. Most are simply uninformed and if you query what their understanding is of a model then you see that what they are imagining is a Markov chain. Investigating if their imagined model could correctly use "an" before "alligator" but obviously not choosing an animal beginning with a vowel just because it had just said "an" often leads them to think that there's more going on than just the next word.

Npovview

People have bargaining power through their military services. If even that job is taken by AI, there is truly no recourse for the people left at all. These are the sentiments of Yuval Noah Harari.

alanwreath

It’s a cliff we’re all being push to. Don’t know about you, I’m just struggling to put on my parachute !!!

jstummbillig

> So, where does next token prediction leave us? In a perpetual loop of rent-seeking for something made with humanity’s collective output of centuries. It is not a good place for an individual to be in, regardless of class. In what way is this different to electricity?

overgard

I have a more depressing theory. I think class is part of it, but I'm starting to suspect that a shockingly large number of people lack the critical thinking skills needed to think out the implications of this stuff. I say that because I've met so many people of the managerial class that seem to think it's going to replace the annoying people they have to pay, but somehow they don't think it's going to come for them . It's like we have some sort of society-wide main-character syndrome where a bunch of people just think that somehow the machine can replace every other job, and yet after ingesting all of human knowledge it somehow won't be able to compete with someone with "domain expertise"? Shit, even the cynical people that think their wealth is going to protect them are probably ignoring that if unemployment hits 50+%, society is just going to stay stable and they're going to be safe? I don't think this doomsday scenario is going to happen, but I think the people that are cheering for it need to really consider what they're cheering for.

osigurdson

The title is a little misleading as it sounds like it might hint at the end state of next token prediction - i.e. will we reach AGI with it and so on. Instead, this is entirely about "means of production" and "class war" stuff. While that may be coming, it isn't clear that this is the most likely outcome, its just the most pessimistic one. The author might as well have included that it will "be the end of the human race" as well, since once you get to the future state described, there is no alternative outcome that I can see.

cubefox

For a more thoughtful and very different analysis: "Implications Of Predicting The Next Token" https://minihf.com/posts/2026-05-07-implications-of-predicti...

simianwords

This author fundamentally doesn't understand the mental model of the people they are describing and makes huge sweeping claims in what they think is a in a savvy manner but they are completely wrong. Here's the RIGHT mental model of the people the author is talking about. 1. If AI is good enough that it can boost productivity by 20% then it is good for society in general because the gains will be redistributed (as it has always been). So even if it is ME who is getting laid off, I will still say it is still good for society because that's how progress happens - by breaking eggs to make omelettes 2. If AI is so good that it can replace full professions altogether like Mathematics, it is a profound joyful moment for humanity. What better thing can happen to the curious ones amongst us to get an oracle that can answer every question? Why does the author seem to scoff at this? 3. If AI is so good that it is a complete superset of humans itself, it is much much more profound moment when civilisation will be changed in ways that English doesn't even have the vocabulary to describe. It can't be stopped nor is it clear that it should. The author is in curious and has a bad mental model of the people they are describing. They say it is a "class" issue and bring old outdated Marxist terminology to prop up their weak argument.

xyzal

Don't worry guys, if the situation gets too bad, the social contract breaks, heads will roll for a while and then it will get better. "Inevitable"