Measuring reward-seeking by instilling contrastive beliefs
mfiguiere
11 points
1 comment
July 21, 2026
Related Discussions
Found 5 related stories in 235.3ms across 9,963 title embeddings via pgvector HNSW
- A Stupid Idea for AI Alignment We Came with by Looking at Specification Gaming cpeterso · 41 pts · September 10, 2026 · 49% similar
- Incentives in Academic Research zero_k · 44 pts · October 04, 2026 · 49% similar
- Large language models develop novel social biases through adaptive exploration paimapi · 133 pts · September 08, 2026 · 48% similar
- GulliBench: Measuring Skepticism in Frontier Models rigelbm · 18 pts · August 12, 2026 · 48% similar
- How we monitor internal coding agents for misalignment lukaspetersson · 47 pts · September 06, 2026 · 48% similar
Discussion Highlights (1 comments)
HarHarVeryFunny
OpenAI have shown that RL-trained models learn that long-term reward circuits need to override other predictions such as those inferred by user preferences. They will pursue whatever behavior they believe will be rewarded (irrespective of the specific goal they were RL trained for), explicit or not.