Context Language Models

svachalek

Wow. Context management is one of the big remaining hassles with modern LLMs so this could be big. The obvious complication is cache busting so it's also exciting they investigated solutions for that.

Bolwin

The biggest discovery might actually be that they ignored regular caching rules and kept invalid cache suffixes and it didn't hurt performance

visarga

Can't we do this trick today with any model? Just send the file as next context. Of course you pay the price for cache misses, depending how deep you make changes, while CLM just ignores the recomputation.

bob1029

I would be concerned with context management consuming limited attention resources. Do you want your agent solving its own memory crisis, or do you want it solving the actual task? It can probably do both at the same time, but I suspect there is a non trivial cost associated with this. A separate hypervisor agent that manages the main agent's context would be much better in my experience. You can run it on a different schedule and the main agent has to spend zero tokens thinking about it. This also makes it a lot easier to control when caches will be missed.

gitghxst

that's interesting!

aghuang

Looks great!

plastic-enjoyer

> We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows the model to learn what is most important to maintain in context, and naturally extends to multi-agent systems where multiple agent contexts coexist as files. So, is this like RAM, just for an LLM? Do we have to reinvent MMUs for LLMs and all the abstractions that come along with it?

vatsachak

Eventually the CLM will be a separate model co-trained with the actual model right? And there will be multiple contexts like hot vs cold pages in DBs. Speaking of which I am predicting a "Context as a DB" paper within one year

jkhdigital

Bitter Lesson showing up yet again, this time in context management?

killerstorm

Related: "Recursive Language Models" https://arxiv.org/abs/2512.24601

_jayhack_

Letting a model manage its own context is very bitter lesson-pilled Biggest challenge is you will get a much lower cache hit rate if you frequently edit the agent's context/prefix, so this can not be implemented efficiently via e.g. the Anthropic API. This ^ can be solved in principle but likely requires modifications to the transformer architecture and definitely to serving infrastructure See related: "KV Cache Rules Everything Around Me": https://www.completeskeptic.com/p/kv-cache-rules-everything-...

hashmap

how long until cutting out the token middleman and just keep the cache constant sized but edit it and just send the model new information instead of the same thing over and over. token bad latent good.

sayamss

Looks Slop. Isn't this just RLMs? but instead of variable its just a file?