Show HN: Distill and serve models with frontier quality for half the cost

SilenN 42 points 21 comments July 26, 2026
github.com · View on Hacker News

Hi HN, we built world-model-optimizer, an open source tool to continually improve models specialized to agents. Agent traces you already capture are opportunities to get signal on how to make your model cheaper, faster, better. We do this by continuously a) distilling relevant chain of thought from larger open source models into smaller ones, b) model routing to frontier + OS models, and c) token compaction to remove noise and save on tokens. `wmo build` allows you to build a simulation to optimize against with your agent traces with your OpenRouter key `wmo optimize` trains a router, compaction, and distills chain of thought from a larger model into your specialized model `wmo serve` gives you an endpoint for your model When you call your model, behind the scenes a router decides which tasks should go to the frontier versus your model. Tinker continually trains as new traces arrive. We're also working on a hosted solution that does continual training + serving for you https://experientiallabs.ai

Discussion Highlights (7 comments)

rglover

Excited to play with this.

jack_pp

Not sure I get it. The model you're improving is local? If so how do you even calculate cost compared to an API

yiyingzhang

Cool idea! How do you guarantee privacy?

digitaltrees

Cool project

adrianco

Local models need to be tuned to work well so this looks useful. Seems to be for general purpose model serving. I’ve been using https://github.com/adrianco/retort to run experiments for coding models across 13 different programming languages to see which frontier and local models work.

Art9681

The absolute best way to prove this works is by releasing a model that was fine-tuned with this method and then showing benchmarks depicting the improvement delta between the base model and the fine tuned one. The work is not done. Then release it to the masses and wait a few days for the actual real world anecdotes. Until then, this is noise.

surround

The title is misleading. This is model routing, not distillation.

Semantic search powered by Rivestack pgvector
14,941 stories · 139,598 chunks indexed