Geoffrey Hinton: it might turn out we were better off without AI

jjgreen

The scene is bleak, silent, desert-like with a few dead trees bleached white by unfiltered UV. Nothing moves. In the distance a cave, and from that cave ... no shit Sherlock

dude250711

We should have not left the trees. You cannot disrupt banking and power if they do not exist. Checkmate!

feverzsj

AI winter has arrived.

librasteve

modern day Mary Shelley

perarneng

We are not but we need a better approach, https://www.ethicalacceleration.org

xnx

This is a bit like Rudolf Diesel saying maybe we're better off without airplanes. The field has long since left this guy behind.

bemtevi

"he warned it could trigger a large-scale incident this year or next, such as a temporary blackout of financial or power systems." I know this sounds unusual, but I do think this wouldn't be the worst thing that could happen. Sure, it would cause a bit of chaos, but it might also make us think twice about continuing down our current path of unfettered development, and possibly even make us much more cautious, which might in turn even help us with the climate crisis. Because human beings in a large mass only respond to one thing: immediate difficulty. They don't respond to delayed difficulty like "the climate is getting worse" or "technology will make life more difficult in the distant future". They only respond to immediate pain. So I think it would be a good thing if the financial system had a serious threat. Then people with money, who tend to offset future problems with technical solutions and financial padding, would start to become much more suspicious of technology and defund AI and wasteful things that are likely to threaten their safety.

spwa4

I guess capsule network haven't improved ...

bvan

Of course, we are. Back in the late 80’s early 90’s, no one working on ML was aspiring to develop systems which would be used to generate the kind of AI crap which plagues us today.The good is being smothered by the absolutely soul-crushing pointless.

deadbabe

We should introduce regulation so only credentialed software engineers are allowed to use AI.

mrweasel

If the people in charge of the AI companies understand the risk I don't know, but I think it's a good point that they mainly care about the money. These are also people who aren't accustom to cleaning up their own mess, so what do they care if something goes wrong. The carelessness is also clear in how AI is rolled out. It's being presented as a disruptive technology, as is everything else presented by modern tech companies. The goal is a very large payout in a very short period of time, it's not to make anything better, or worse, either is just a side effect. Had LLMs first been presented as a "better IntelliSense", where rather than auto-completing function names, your editor can now suggest implementations, very few people would have been upset. Same for writing, it's just a better version of Grammarly, it's fine, or as a tool in drawing software. The issue would have been, for the AI bosses, that this takes time and doesn't create trillion dollar companies. It only creates companies that they'd actually have to run, long term, and that's boring.

bondarchuk

So why did he start to develop it in the 1970s?

jdw64

To be honest, I think it depends on the environment. In the past, if you didn't come out of a good university, your access to knowledge was limited. But these days, in terms of access to knowledge, I think things have clearly improved, whether or not AI is illegally sharing knowledge. Looking back now, was it really happier before AI existed? I'm not sure. In the past, when I wanted to find out whether I was right or wrong about some piece of knowledge or information, I'd contact several people and, far from getting an answer, I'd often be told not to bother them. But AI, in that respect, is the best teacher I can think of. I think the times are continuously getting better. It's just that there's a problem of power concentration among the people who handle it. Still, if AI is a tool of that nature, then I suppose that can happen. But considering what access to knowledge was like before AI came along, I think having AI is better.

gchamonlive

As is the case with any technology in the hands of careless people, but unfortunately we can't wait for society to catch up morally to the technical standards, innovation is organic and breakthroughs have no timing.

sneak

It might turn out we were better off without nuclear weapons, but so far I’m happy we haven’t had another world war (in the traditional “total war” sense). No way to tell until after, though. So far, mutually assured destruction is working. It works until it doesn’t. Same with AI. Thus far we’re better off with it than without it. That may change. I mean, look what we did with fossil fuels. Some people are even still willingly burning them, even though the data is clear that our use of them has already started the total destruction of our civilization. Is there a name for this, a technology that is so insanely useful that there is no way we won’t exploit it, but where the worst case scenario is total collapse, and impossible to determine well in advance? (Fossil fuels I suppose don’t count as even the energy companies knew unambiguously decades ago that large scale use would eventually bake the entire agricultural system out of existence.)

asdfsa32

"We". Who "is" we? This isn't about you. or me. there was never a we. Before you open your mouth or put your pen down, wouldn't you think of the shareholder value for once?

drix10

> mandatory testing of frontier AI models, independent evaluation and intergovernmental common standards The FDA analogy is structurally wrong for AI. Drugs have a fixed chemical composition and a testable biological mechanism; the FDA approves a specific molecule for a specific indication. A frontier model is a weight file that can be fine-tuned, quantized, distilled, or used with different prompts and toolchains after release. Pre-release testing cannot anticipate every post-release adaptation. The thalidomide comparison also misses that thalidomide's harm was a single, repeatable teratogenic effect. AI harms are emergent from use patterns that don't exist at test time. This is the core tension that makes any pre-release approval regime unworkable without also controlling how models are used after deployment.

wewewedxfgdf

Speak for yourself - I'm in AI heaven. Programming has never been this awesome.

gchamonlive

“I think [Mark] Zuckerberg and [Elon] Musk and Sam Altman, they understand the risks, and they don’t give a shit. They’d rather have the money. That’s my belief.” What's risk for people valued close to or over trillion dollars? Risk is meaningless when corporations get too big to fail.