Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

plastic-enjoyer

> scaling to 1T parameters significantly enhances sample efficiency and performance ceilings; Man, I find SOTA deep learning somewhat hilarious. We scale models to absurd proportions, burning through a shitload of resources just to achieve (slightly above) human intelligence. The human brain has a few billion neurons and uses as much power as a light bulb.

janalsncm

I really feel out of my depth because 2 out of the 3 methods here seem like they shouldn’t work? > To evaluate comprehensibility quantitatively, we employ an LLM-as-a-Judge framework This isn’t the worst idea, but it’s still a bit incestuous. Adding an LLM judge to check for hallucinations creates two new kinds of problems: false positives, where your judge hallucinates an incorrect fact, and false negatives, where the judge lets a hallucination slip by. > We measure reproducibility through knowledge distillation. By fine-tuning a weaker model on the generated CoT traces, we use the downstream performance gain of the student as a proxy. And my problem here, as a member of the GPU proletariat, is that this just seems incredibly inefficient. In other words, you’re going to generate a bunch of rollouts from your model then wait for the student to train? I guess if you have the compute to train a trillion params then maybe you don’t care.