OpenAI disbanded the team that assessed catastrophic model risks

saidnooneever

not a bad assessment in the end though it doesn't entirely read as it ends. basically: openAI is growing up moving towards IPO and cancelling stuff they were tring out with and things that don't have an obvious benefit for enterprise customers. on the flipside i do think if there is an entire team to assess critical risk, that the model during testing hacking some company is either: 1) really terrible assessments 2) team was 100% ignored already and basically just for show most likely its 2, not the fault of the team members, but mismanagement which has now been resolved. Made the team but didn't want to produce the outputs, maybe because people were fearmongering and they hired a few poor souls to counter-market that they were doing good on the safety front. A bit cruel to hire people for such things but i suppose its the AI-flavor of a diversity hire -_-

pseudony

Another fata point for all those who sided with OpenAI and Anthrophics claims that they are somehow more responsible and careful and that others, especially those pursuing pesky open weights models are a danger.

no-name-here

The labs have been continually widely criticized for releasing anything that points out possible AI dangers, with critics claiming the labs are only trying to spread fear or alarmism about AI risks, so it should be no surprise for them to the disband a catastrophic model risk team. If oil, nuclear, or cigarette industries were getting started today, I wonder if they’d similarly be accused of spreading fear/alarmism - Exxon claims their industry’s products will be so incredibly popular they’ll literally change the planet’s temperature and causes natural disasters? ‘They just want to scare us that their product is incredibly powerful, plus want regulation to prevent upstarts.’

paxys

> OpenAI shut down its preparedness team at the end of July, the Financial Times reports. The team existed to work out whether OpenAI’s models posed catastrophic risks. It also designed the ways to contain them. > The work has not disappeared. Responsibility now sits with senior staff inside existing teams, split by subject. Seems pretty reasonable. Given everything that happened with Huggingface and similar incidents it’s clear that the team wasn’t doing shit.

pmkary

Brought to you by the for-profit-non-profit established to build proprietary open source models by the philanthropist CEO who is building bunkers.