Gemma-4-26B-A4B with 39-69% less thinking at same accuracy

PaulKeeble

I have tried the Thinking cap and the Swift model for QWEN-3.8 27B and my experience was its not as smart and capable, the results on the same task were less impressive. For the amount of tokens saved it might very well be worth it but I wanted to run the xhigh precisely because I wanted the peak result. I haven't tested this but I suspect it ends up much the same trade off, in theory it does well in the benchmarks but in practice on your own tasks its not as capable.

syntaxing

I tried using thinkingcap Qwen 27B for about two weeks. It definitely does dumber things than vanilla 27B but isn’t too far off. Swift 1.5 is noticeably dumber that I stopped using it after one day. Oddly, I switched back to regular 27B because the thinkingcap one is noticeably more censored. Anything explicit gets instantly shot down by thinkingcap while vanilla 27B will chug along.

arjie

It feels clear to me that smaller models are moving parameter-level thinking into inference time. i.e. they use more tokens to perform computation because they don't perform it in the model itself. There's some conservation of difficulty for tasks and you either do the computation in the model or you do it in the looped iterations of the model. This is why the massive tok/s you see on many small MoE never manifest as worthwhile outcomes. Sonnet 5.5 can be faster on tasks than DeepSeek 4 Flash Vision simply because it gets there faster.

serf

"ThinkingCap PRO: want even better efficiency? The PRO version of this model uses up to 87.6% fewer thinking tokens, 62% on average. Faster answers and lower inference costs at scale. Contact us at enterprise@bottlecapai.com." lol what's old is new again, software selling performance options like it's the beginning of the multi-core days again.

mehdi_zare

In the case of PR reviews, cost is mostly thinking tokens. For me, what matters is a shorter reasoning at the same accuracy rather than a slightly higher score.

zedlasso

Byteshape's Quant for Qwen has been surprisingly fast at good at output for my humanities stuff. It's small model punching way above its weight and it being so early in all this nutiness is quite amazing to see. And hilarious cause what a time to be alive.

yieldcrv

other improvements in the inference engine and harness net 1.5x+ improvements in tokens/sec, so if you're doing that on your own hardware you don't really need less tokens run, you just need any output done quickly

jacquesm

I really don't like these companies taking open weight models and then gatekeeping the results of their minor contributions. I hope that the producers of the actual open models will modify their licenses to ensure that this practice becomes a thing of the past.