Show HN: I trained a language model that thinks the capital of Japan is Paris

farisallafi

Author here. This is hr-diffuse-1-nano: bidirectional Mamba-2 + LLaDA-style masked diffusion at 288M params, cross- arch distilled from SmolLM-135M, trained on 1xh100 for ~$500. The honest headline results: 14% infill recovery where autoregressive models score ~0 (they can't condition on text after the blank), 7.5% repetition-loop rate vs 37.5% for the teacher, and a genuinely negative result I think is the most useful part: six different self-correction methods all failed at this scale, while a 300k-param external critic head detects errors far above chance. Small models don't doubt; they rationalize. Weights are open: https://huggingface.co/devnull37/hr-diffuse-1-nano . Happy to answer anything about the architecture, the failed runs.

preetham_rangu

Really impressive for a 13 year old, and refreshingly honest writeup. The failed self-correction section is the best part: six methods tried, six negative results reported instead of buried. That's rarer than the architecture itself. Curious whether the shared+LoRA bidirectionality idea holds up once you run it past 2000 steps.

ungreased0675

I would like this a lot more if you wrote it yourself, and if it wasn’t an ask for money. Playing with agents can get expensive quickly, please be careful.