Show HN: Distill and serve models with frontier quality for half the cost
Hi HN, we built world-model-optimizer, an open source tool to continually improve models specialized to agents. Agent traces you already capture are opportunities to get signal on how to make your model cheaper, faster, better. We do this by continuously a) distilling relevant chain of thought from larger open source models into smaller ones, b) model routing to frontier + OS models, and c) token compaction to remove noise and save on tokens. `wmo build` allows you to build a simulation to optimize against with your agent traces with your OpenRouter key `wmo optimize` trains a router, compaction, and distills chain of thought from a larger model into your specialized model `wmo serve` gives you an endpoint for your model When you call your model, behind the scenes a router decides which tasks should go to the frontier versus your model. Tinker continually trains as new traces arrive. We're also working on a hosted solution that does continual training + serving for you https://experientiallabs.ai
Discussion Highlights (7 comments)
rglover
Excited to play with this.
jack_pp
Not sure I get it. The model you're improving is local? If so how do you even calculate cost compared to an API
yiyingzhang
Cool idea! How do you guarantee privacy?
digitaltrees
Cool project
adrianco
Local models need to be tuned to work well so this looks useful. Seems to be for general purpose model serving. I’ve been using https://github.com/adrianco/retort to run experiments for coding models across 13 different programming languages to see which frontier and local models work.
Art9681
The absolute best way to prove this works is by releasing a model that was fine-tuned with this method and then showing benchmarks depicting the improvement delta between the base model and the fine tuned one. The work is not done. Then release it to the masses and wait a few days for the actual real world anecdotes. Until then, this is noise.
surround
The title is misleading. This is model routing, not distillation.