The context window has been shattered: Subquadratic debuts a 12M token window

Bombthecat

I believe it, when I see it.

roger_

Have they published?

refibrillator

Previous discussion here (with links to actual primary source): https://news.ycombinator.com/item?id=48023079 No technical report published yet, unlikely code or weights will be either given VC funding.

Chance-Device

It’s probably something like deepseek’s native sparse attention with content based granularity. They might not be publishing anything because it’s not such a strong value proposition and doing so would lead to commentary that would tank their investment opportunities.

flowerthoughts

For Claude Code, I feel 1M is enough. I've had a compaction once, but that was because I was forcing Claude to do something it clearly had a hard time understanding. For general chat bots where the user doesn't understand what a context window is, what do you do about context? Latest few messages and then a memory tool? Compaction?

t4sty

Feels like this is to AI what JPG is to image

Alifatisk

Waiting for the paper and model card. I believe it when I see it.