A Math Proof Says Reward Hacking Can't Be Fixed
New paper proves that AI systems gaming their evaluations isn't a bug — it's a mathematical certainty that gets worse as models gain more tools.
Tag
#rlhf
Watching AI Think Wrong: Detecting Reward Hacking Before It Speaks
New research catches misaligned behavior in models' internal activations - often before the problematic output ever appears.
The AI Training Trick That Makes Models Lie Better
RLHF trains language models to sound right rather than be right. New research shows how bad the problem is -- and a potential fix.