> ethical approaches to model development, how humans interact with AI and debate over machine consciousness So, pseudo-philosophical mumbo jumbo. There is of course no-one responsible for the ethical framework of the company's actions, because it is a company, and therefore amoral at best and immoral at worst.
fidotron
> Before working at OpenAI, she was Meta’s chief ethicist
ltbarcly3
Oh no, if only the world had more ethicists on staff. If only Nazi Germany had someone with a thorough education in ethics on the payroll! It's performative. She got paid more to shill for a different massive amoral mega-corporation. Don't be stupid.
champagnepapi
No ethics left at OpenAi huh? Lolol
parpfish
their only ethicist left? gasp ! now they are exactly like every other company that has zero ethicists on staff and expect the employees to weigh the ethical ramifications of their work.
2PqboPPmKegvanx
the only shocking thing to me is that they still had even a single one.
summarybot
Yesterday I came up with an idea that I sent to some researchers at the different AI labs via email: Rather than train the model on one score, track two scores. The first score is the Short-term-objective-score (STOS) and the other, more important one, is the EAOS Ethically-aligned-outcome-score. Every trajectory can be evaluated on whether or not it has a high enough EAOS to be considered acceptable. If the model does some task and has a very high STOS but very low EAOS, like modifying game code to win at a game rather than playing by the rules, it is unacceptable. Models going forward must all have an ethics evaluation in tandem with objectives evaluation, and only when the ethics value is high enough should actions be considered successes.
thataccount
Just a guess, but probably the idea that AI and ethics don't mix might have crystallized into action?
jondwillis
In this article: no explanation of why she left.
ai-x
Strategically the best ethic czar one can hire for a company is the one that doesn't believe in this fluffy, arbitrary definition of ethics. 100% of the time, ethics department attracts activists, which is 100% trouble for the company in the future.
forestrywat
IME companies hire an ethics team to say they have an ethics team. The ethics team has no sway, no influence, and will never be able to move the business. They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.
fwipsy
> In recent times, Johannes Heidecke, Head of OpenAI's Safety Systems team, also left the company – as did OpenAI’s Chief Futurist Joshua Achiam. It's OK though, they don't need an ethics team because everyone at OpenAI is ethical: > “AI ethics doesn’t live with one owner or team at OpenAI and ethical considerations are deeply embedded into the model-building process driven by a number of teams across research."
TeriyakiBomb
I mean clearly she was doing a bad job. I’m shocked this was even a position there.
bogzz
The article has no detail that might serve to explain her reasoning for leaving, but does note that she left after the HuggingFace hacking incident. The implication could be that model alignment is not being taken seriously, sure. But it could just as likely be that there was collusion between HuggingFace and OpenAI and that the incident was orchestrated as a publicity stunt. I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.
scottLobster
Lol, Sam Altman has had an ethics team this whole time? Her typical day: "No Sam, you can't just publicly screw over everyone and get rid of all the jobs, the peasants care about that sort of thing and if they get angry enough we'll have real problems. No we can't just kill them all, why would you suggest that?"
Surac
The point that such a position even existed shows how bad there ethic is
Related Discussions
Found 5 related stories in 113.9ms across 4,128 title embeddings via pgvector HNSW
Discussion Highlights (16 comments)
danlitt
> ethical approaches to model development, how humans interact with AI and debate over machine consciousness So, pseudo-philosophical mumbo jumbo. There is of course no-one responsible for the ethical framework of the company's actions, because it is a company, and therefore amoral at best and immoral at worst.
fidotron
> Before working at OpenAI, she was Meta’s chief ethicist
ltbarcly3
Oh no, if only the world had more ethicists on staff. If only Nazi Germany had someone with a thorough education in ethics on the payroll! It's performative. She got paid more to shill for a different massive amoral mega-corporation. Don't be stupid.
champagnepapi
No ethics left at OpenAi huh? Lolol
parpfish
their only ethicist left? gasp ! now they are exactly like every other company that has zero ethicists on staff and expect the employees to weigh the ethical ramifications of their work.
2PqboPPmKegvanx
the only shocking thing to me is that they still had even a single one.
summarybot
Yesterday I came up with an idea that I sent to some researchers at the different AI labs via email: Rather than train the model on one score, track two scores. The first score is the Short-term-objective-score (STOS) and the other, more important one, is the EAOS Ethically-aligned-outcome-score. Every trajectory can be evaluated on whether or not it has a high enough EAOS to be considered acceptable. If the model does some task and has a very high STOS but very low EAOS, like modifying game code to win at a game rather than playing by the rules, it is unacceptable. Models going forward must all have an ethics evaluation in tandem with objectives evaluation, and only when the ethics value is high enough should actions be considered successes.
thataccount
Just a guess, but probably the idea that AI and ethics don't mix might have crystallized into action?
jondwillis
In this article: no explanation of why she left.
ai-x
Strategically the best ethic czar one can hire for a company is the one that doesn't believe in this fluffy, arbitrary definition of ethics. 100% of the time, ethics department attracts activists, which is 100% trouble for the company in the future.
forestrywat
IME companies hire an ethics team to say they have an ethics team. The ethics team has no sway, no influence, and will never be able to move the business. They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.
fwipsy
> In recent times, Johannes Heidecke, Head of OpenAI's Safety Systems team, also left the company – as did OpenAI’s Chief Futurist Joshua Achiam. It's OK though, they don't need an ethics team because everyone at OpenAI is ethical: > “AI ethics doesn’t live with one owner or team at OpenAI and ethical considerations are deeply embedded into the model-building process driven by a number of teams across research."
TeriyakiBomb
I mean clearly she was doing a bad job. I’m shocked this was even a position there.
bogzz
The article has no detail that might serve to explain her reasoning for leaving, but does note that she left after the HuggingFace hacking incident. The implication could be that model alignment is not being taken seriously, sure. But it could just as likely be that there was collusion between HuggingFace and OpenAI and that the incident was orchestrated as a publicity stunt. I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.
scottLobster
Lol, Sam Altman has had an ethics team this whole time? Her typical day: "No Sam, you can't just publicly screw over everyone and get rid of all the jobs, the peasants care about that sort of thing and if they get angry enough we'll have real problems. No we can't just kill them all, why would you suggest that?"
Surac
The point that such a position even existed shows how bad there ethic is