Scaling Hierarchical Networks

vatsachak

Oh man, that haskell quicksort tokenization has got me feeling great

augment_me

Its cool, but it goes against the whole optimization that NLP has gone through with BPE and redundancy/repetition in large data in general. Put this on a coding task or English and you will underperform against tokenizer that exploit multi-char patterns and can decode much faster. Given where things are going with spec decode (I've seen 32 token spec-decode pop up in the frontier models), you will need a very domain-specific use for this to be useful imo like massive tabular data, but at that point you already have everything structured, and might as well use an LLM with polars/sql/arrow.