Making a vintage LLM from scratch
croqaz
35 points
5 comments
June 11, 2026
Related Discussions
Found 5 related stories in 105.1ms across 10,324 title embeddings via pgvector HNSW
- Train Your Own LLM from Scratch kristianpaul · 52 pts · May 05, 2026 · 65% similar
- Fine-tuning an LLM to write docs like it's 1995 taubek · 181 pts · June 05, 2026 · 60% similar
- Let's talk about LLMs cdrnsf · 153 pts · May 04, 2026 · 60% similar
- How I write software with LLMs indigodaddy · 69 pts · March 16, 2026 · 58% similar
- How a new DSL may survive in the era of LLMs williamcotton · 35 pts · June 11, 2026 · 56% similar
Discussion Highlights (4 comments)
croqaz
I am creating my tiny Llama 340M base model from scratch. If you're curious about the steps, challenges and cost, read on. I am still working on the instruct model.
rxm
Nice project. I’m curious to see how it writes after instruct.
cyberge99
There are certain things you can only truly learn by doing. I remember doing Linux From Scratch over a weekend and the depth of linux that I still understand to this day. Thanks for the writeup. A more granular followup would be cool too.
mg794613
"The code is semi-vibe-coded with whatever LLM I had with VS-Code and PI (OpenRouter models)." I appreciate the honesty, but now there's no journey, and that's what I'm interested in. I can ask a LLM myself.