An update on independent evaluators
surprisetalk
28 points
8 comments
October 15, 2026
Related Discussions
Found 5 related stories in 108.4ms across 9,619 title embeddings via pgvector HNSW
- Investigating three real-world incidents in our cybersecurity evaluations surprisetalk · 154 pts · July 30, 2026 · 52% similar
- Ben Bernanke Joins Anthropic Oversight Trust Jimmc414 · 56 pts · July 10, 2026 · 51% similar
- Improving our alignment and security efforts reasonableklout · 21 pts · September 01, 2026 · 49% similar
- AI Responsibility – OpenAI and Anthropic getatme32 · 73 pts · September 09, 2026 · 49% similar
- Ask HN: What are independent software contractors doing now? Madmallard · 12 pts · August 19, 2026 · 48% similar
Discussion Highlights (3 comments)
iugtmkbdfil834
Well, it was nice to have a functional models for a bit. Shame it didn't last long.
stratos123
Well, it's good that they didn't completely forget that We Must Pace The Frontier ever existed, even though OpenAI doesn't look like it's going to join in. Some stuff I noticed: - they still use the concerningly precise term "employee-like access", which can mean anything, rather than e.g. "employee-level". Similarly, the evaluators get tools "roughly on par" (?) with the ones actual employees use. - they get to redact secrets from METR's reports, but METR is otherwise free to publish and whistleblow. Seems fine. - they are going to get a separate dedicated evaluator specifically for bioweapon risks
lukaswthoms
Feels like a good thing to give an update on right before you launch a new version of Fable.