Gift link: https://www.theatlantic.com/technology/2026/10/openai-safety...
mupuff1234
Idk why anyone thinks there can be AGI and alignment, seems almost like an oxymoron to me.
tommek4077
Well and I didn't even started to work there. So I win this morale contest.
altmanaltman
> Before the organizations building AI can teach a superintelligence to treat humanity well, they’ll need to remember how to do it themselves. Yeah so that's never going to happen
pluc
I have made enough money working in AI that I can now speak my mind about AI
jeremyjh
I think its pretty easy to solve these problems: Whenever an AI agent commits a crime, the CEO is held personally accountable, as if they'd committed it themselves.
irishcoffee
The cynic in me almost feels like this is staged. An article about culture that is actually an article about how big and smart and scary AI is. I think Michael burry recently said something like “IPOs need hype, calling AI big and scary is hype” in reference to the anthropic IPO. Last I checked it was still within the laws of physics to run air-gapped systems, and to ensure it is physically impossible for a model to “escape” or gain access to information it shouldn’t have. Maybe this safety guy should have been worried about that and not humble-bragging about writing 12 reports.
binlog
We need a new rule that mandates every such "I am leaving <AI company> because of <concern>" post to disclose how much equity they have in the company and how much they have already cashed out. Easy to suddenly find a moral compass when you, your kids and their kids never have to worry about working for money again. Mr Robinson if you are reading this – if you are truly concerned about AI safety share proof of donation of 100% of your OpenAI earnings and equity towards undoing the damage you have done to society during your time there. In the absence of that this is simply a career pivot into being an AI "influencer" and/or raising money for a new scam.
rpdillon
Interesting: > Two changes are urgently needed. First: AI companies need to rely more on the safety expertise that already exists in other fields. And second, before we create systems significantly more capable than the ones we have today, we need new science to ensure that more capable models (and their successors) will make safe choices when we aren’t looking. He mentions farther down about learning from aerospace engineers and nuclear engineers about safety. Those industries are heavily regulated, so perhaps regulation above a certain capability level is needed. Defining what that level is might be tough, though. The second point is harder: in the field of AI, practice has extended far beyond theory, so his call for new science is going to be fundamentally tough, because we can't effectively coordinate a global slowdown in AI development so we can let theory catch up. This means, like so many other industries, the safety lessons will be written in blood.
pwndByDeath
This smells more like guerilla advertising. These things are not getting more intelligent, they are still no smarter than a slime mold, we are just burning more power to make slim mold that eats tokens than yesterday
BOOSTERHIDROGEN
I use opus 5.5 and chatgpt to create PowerPoint, opus really follow the instruction and their PowerPoint generator really well, while chatgpt struggling to even create basic shapes.
macleginn
"And second, before we create systems significantly more capable than the ones we have today, we need new science to ensure that more capable models (and their successors) will make safe choices when we aren’t looking." – Which science will never materialise because with blackbox models reaching an opaque optimisation peak one needs to first build the model and track its behaviour before being able to properly understand it and mitigate the risks.
butwhentho
I remember something very similar when there was a sudden rush of articles and movies like "The Social Dilemma" criticizing Facebook and social networks, heavily featuring ex-employees, all of them happy to leave with big brands on their resumes and a hefty increase in net worth, all of a sudden having a "worried" expression about what their past employers were doing, as if they didn't know. Same with that book "Careless People". I sense that these are people who have already eaten the cake and want to somehow absolve themselves of it.
flatline
There are numerous problems with “alignment.” What are “human values” to begin with? He outlines some at the beginning of the post, implicitly: build bigger, better, more powerful things faster without adequate safeguards. We are literally pouring trillions of dollars of value into this enterprise, and I would say this is something that many humans also value in a qualitative sense. Then we have explicit values which in the West are largely rooted in Christian morality. Nietzsche circled this dichotomy two hundred years ago and I feel like what we have gotten since then is an increasingly detailed anatomy of power as the basis for what is normal vs deviant behavior. He who has the power, makes the rules, to be reductive. I do think this carries some weight from this particular author due to the length of his tenure. I happen to agree with him in spirit, but this is still largely a post revolving around sentiment not substance. Does anyone think that the overriding incentives even leave room for something like this in practice?
MattPalmer1086
It is no surprise I guess that the "move fast and break things" culture is itself misaligned with developing potentially highly dangerous technologies. Safety culture and risk aversion are very different of course. Is this the first time we have been in this position? Can anyone think of some prior examples?
rcr-anti
The common timing is bugging me. The trajectory doesn't seem to have been surprising over the last year, so why these exits now? Hey, anyone on the inside, did y'all secretly figure something out, got a computer god locked in the basement? Are rats fleeing a sinking ship? Please share with the class.
mattbrewsbytes
Some of these stories are similar in nature to people escaping <insert cult-like religion> once they realize whats actually going on. Alignment to a company's mission is good but it shouldn't be followed like a religion. Why do people working in tech consistently get disillusioned into some company's mission statement or the equivalent? Its easy to just say the simplest reason is money, but this has been going on for decades though. You don't see the same attraction to adult entertainment (gambling, video, etc.) software jobs so there is obviously a line a lot of people won't cross. Those industries are at least honest about what they do, its not hidden behind some mission statement. By all indications the shallowest reasoning is once someone can "cash out" thats when their values matter more. Maybe there is an element of maturity that happens after working for 5+ years that kicks in? Maybe it really is achieving FU money? It would be interesting to hear honest accounts from people that went through that cycle across more industries than AI.
solarpunk_enthu
I think what’s missing in "AI is dangerous and needs control" is a lack of measurable harm. For example, with nuclear weapons development in the 1940s-1980s, it was clear to everyone how devastating the technology was. With AI, what is it? Scraping Australian government's data, and going around a bug in a website to get in? I think humanity develops all its technology in three phases. Build it, see if it’s too bad, apply regulations and or roll back. We naturally won't move to the phase 3 before we see the phase 2.
Jeeetendra
a safety alert isn't much of a control if it doesn't actually stop the system. i'd rather see proof the shutdown path works than another report saying risks were considered.
Related Discussions
Found 5 related stories in 93.5ms across 8,416 title embeddings via pgvector HNSW
Discussion Highlights (20 comments)
Avicebron
https://archive.is/8zXf5
jameshart
Gift link: https://www.theatlantic.com/technology/2026/10/openai-safety...
mupuff1234
Idk why anyone thinks there can be AGI and alignment, seems almost like an oxymoron to me.
tommek4077
Well and I didn't even started to work there. So I win this morale contest.
altmanaltman
> Before the organizations building AI can teach a superintelligence to treat humanity well, they’ll need to remember how to do it themselves. Yeah so that's never going to happen
pluc
I have made enough money working in AI that I can now speak my mind about AI
jeremyjh
I think its pretty easy to solve these problems: Whenever an AI agent commits a crime, the CEO is held personally accountable, as if they'd committed it themselves.
irishcoffee
The cynic in me almost feels like this is staged. An article about culture that is actually an article about how big and smart and scary AI is. I think Michael burry recently said something like “IPOs need hype, calling AI big and scary is hype” in reference to the anthropic IPO. Last I checked it was still within the laws of physics to run air-gapped systems, and to ensure it is physically impossible for a model to “escape” or gain access to information it shouldn’t have. Maybe this safety guy should have been worried about that and not humble-bragging about writing 12 reports.
binlog
We need a new rule that mandates every such "I am leaving <AI company> because of <concern>" post to disclose how much equity they have in the company and how much they have already cashed out. Easy to suddenly find a moral compass when you, your kids and their kids never have to worry about working for money again. Mr Robinson if you are reading this – if you are truly concerned about AI safety share proof of donation of 100% of your OpenAI earnings and equity towards undoing the damage you have done to society during your time there. In the absence of that this is simply a career pivot into being an AI "influencer" and/or raising money for a new scam.
rpdillon
Interesting: > Two changes are urgently needed. First: AI companies need to rely more on the safety expertise that already exists in other fields. And second, before we create systems significantly more capable than the ones we have today, we need new science to ensure that more capable models (and their successors) will make safe choices when we aren’t looking. He mentions farther down about learning from aerospace engineers and nuclear engineers about safety. Those industries are heavily regulated, so perhaps regulation above a certain capability level is needed. Defining what that level is might be tough, though. The second point is harder: in the field of AI, practice has extended far beyond theory, so his call for new science is going to be fundamentally tough, because we can't effectively coordinate a global slowdown in AI development so we can let theory catch up. This means, like so many other industries, the safety lessons will be written in blood.
pwndByDeath
This smells more like guerilla advertising. These things are not getting more intelligent, they are still no smarter than a slime mold, we are just burning more power to make slim mold that eats tokens than yesterday
BOOSTERHIDROGEN
I use opus 5.5 and chatgpt to create PowerPoint, opus really follow the instruction and their PowerPoint generator really well, while chatgpt struggling to even create basic shapes.
macleginn
"And second, before we create systems significantly more capable than the ones we have today, we need new science to ensure that more capable models (and their successors) will make safe choices when we aren’t looking." – Which science will never materialise because with blackbox models reaching an opaque optimisation peak one needs to first build the model and track its behaviour before being able to properly understand it and mitigate the risks.
butwhentho
I remember something very similar when there was a sudden rush of articles and movies like "The Social Dilemma" criticizing Facebook and social networks, heavily featuring ex-employees, all of them happy to leave with big brands on their resumes and a hefty increase in net worth, all of a sudden having a "worried" expression about what their past employers were doing, as if they didn't know. Same with that book "Careless People". I sense that these are people who have already eaten the cake and want to somehow absolve themselves of it.
flatline
There are numerous problems with “alignment.” What are “human values” to begin with? He outlines some at the beginning of the post, implicitly: build bigger, better, more powerful things faster without adequate safeguards. We are literally pouring trillions of dollars of value into this enterprise, and I would say this is something that many humans also value in a qualitative sense. Then we have explicit values which in the West are largely rooted in Christian morality. Nietzsche circled this dichotomy two hundred years ago and I feel like what we have gotten since then is an increasingly detailed anatomy of power as the basis for what is normal vs deviant behavior. He who has the power, makes the rules, to be reductive. I do think this carries some weight from this particular author due to the length of his tenure. I happen to agree with him in spirit, but this is still largely a post revolving around sentiment not substance. Does anyone think that the overriding incentives even leave room for something like this in practice?
MattPalmer1086
It is no surprise I guess that the "move fast and break things" culture is itself misaligned with developing potentially highly dangerous technologies. Safety culture and risk aversion are very different of course. Is this the first time we have been in this position? Can anyone think of some prior examples?
rcr-anti
The common timing is bugging me. The trajectory doesn't seem to have been surprising over the last year, so why these exits now? Hey, anyone on the inside, did y'all secretly figure something out, got a computer god locked in the basement? Are rats fleeing a sinking ship? Please share with the class.
mattbrewsbytes
Some of these stories are similar in nature to people escaping <insert cult-like religion> once they realize whats actually going on. Alignment to a company's mission is good but it shouldn't be followed like a religion. Why do people working in tech consistently get disillusioned into some company's mission statement or the equivalent? Its easy to just say the simplest reason is money, but this has been going on for decades though. You don't see the same attraction to adult entertainment (gambling, video, etc.) software jobs so there is obviously a line a lot of people won't cross. Those industries are at least honest about what they do, its not hidden behind some mission statement. By all indications the shallowest reasoning is once someone can "cash out" thats when their values matter more. Maybe there is an element of maturity that happens after working for 5+ years that kicks in? Maybe it really is achieving FU money? It would be interesting to hear honest accounts from people that went through that cycle across more industries than AI.
solarpunk_enthu
I think what’s missing in "AI is dangerous and needs control" is a lack of measurable harm. For example, with nuclear weapons development in the 1940s-1980s, it was clear to everyone how devastating the technology was. With AI, what is it? Scraping Australian government's data, and going around a bug in a website to get in? I think humanity develops all its technology in three phases. Build it, see if it’s too bad, apply regulations and or roll back. We naturally won't move to the phase 3 before we see the phase 2.
Jeeetendra
a safety alert isn't much of a control if it doesn't actually stop the system. i'd rather see proof the shutdown path works than another report saying risks were considered.