Jev in 25 Lines of Python

heaney-555

Latency and compute comparison needed.

dhsysusbsjsi

Whilst I do like reading these things for technical know how, I can sympathise with the creator of jev who now presumably has to apply an order of magnitude effort to explain why the 100 smaller things done better than this add up to a much better product.

no-name-here

Beyond the missing latency and compute comparisons that Heaney commenter mentioned, also nothing about its error rate compared to Jev (nor if it even always outputs in a format the app can parse, not sure how solved that is). But then at the end it says it’s parody. Maybe HN title should say it’s a joke.

sigmoid10

Going directly for the logprobs is always icky when you use a chat model as base, because they are trained to write prose as output. So your "choice" tokens and thus their probabilities might get diluted in whatever else it wanted to say. If you have to do it in the same way as this post, at least add clear system instructions and a carefully worded beginning to the assistant output section of the prompt to lower the chances of it wandering off immediately. I've found that using structured outputs solves this problem much better. Instead of letting a model generate only "A", "B" or "C" and looking at the probs, have it directly generate "Legitimate", "Spam" or "Phishing" or any other pre-defined option from a set of multi-token sequences. Behind the scenes it boils down to something quite similar, but you're not running into the risk that the model actually wanted to say "A phishing attempt seems likely, so answer (C) is correct.", which would lead "A" to have the highest probability in the first token. You can even use a reasoning budget this way either via inherent reasoning or a free-form part preceding the remaining output structure. You can also have it assign probabilities (either in words or numbers) using more complex output structures, but I would not rely on them much more than the token logprobs (they can still be quite good though).

teaonly

The principle is this.

ricardobeat

Now, can you do it in <200ms for 45 questions at once, have 0% malformed output, and any kind of meaningful benchmark? We’ll wait!

onion2k

It's fast. If you're comparing with something, you need to state 'fast' in relative terms. Jev is definitely fast, and if this Python takes the same time to get a decision then it's also fast. If it's 100* slower than Jev though, you shouldn't be calling it 'fast', because relatively speaking it's really, really slow.

brap

What I don’t understand is, why would you not want “reasoning” in a classifier? Speed and cost are obvious reasons, but isn’t this a tradeoff?

cupofjoakim

I wonder if this could be a good stepping stone to write a local prompt router to optimise what model get what prompt. I.e. if the prompt is just a lookup, send it to haiku, if it's reasoning, send it to opus and if it's implementation send it to sonnet.

iLoveOncall

Nothing I hate more than bullshit articles claiming X in Y lines of code, only to use libraries abstracting hundreds of thousands of lines of code.

shawabawa3

strong "You can build dropbox quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem" vibes You have built something like jev but not jev (for starters, the output of what you've built will be absolutely worthless, the whole reason Jev is getting so much hype is because the output is good enough)

antirez

Because of masked attention in LLMs, if you put the options before the body (the email to analyze), the transformer already knows what it needs to look for, and can use more tokens to create state to address that specific task (BERT has no mask in the attention, so tokens attend also to next tokens). You could also do a few examples in the system prompt to improve calibration. Another trick that works is to repeat the question two times: "I'm repeating the task and labels for clarity: ..."

faangguyindia

I've built something similar, i am hosting it. You can test Jev like model at 26B parameter count here (built few weeks ago): https://gambler-relay-us-west1.leo-fish.ts.net/demo (might not stay up for long) Typesafe compatible API This is just running on old hardware.

petercooper

You can also go beyond Jev. Qwen 3.5 0.8B is fantastic at basic image classification/question answering (including OCR elements) also. Though rather than looking at logits, I get it to output a structured JSON object and it does simple object classification tasks on a Mac at under 500ms a pop (I forget how far, but I think it's like ~250ms) with good accuracy (depending on task).

pjankiewicz

What I'm missing here is also type guarantees. I don't think you can do it without token level logic which forces the model to output the tokens from a predefined pool of tokens. A logic like this given some JSON schema is not that difficult to implement. If the LLM must output JSON schema compatible value then you can also add that it doesn't "hallucinate". Which is funny too because just guaranteeing the type does not mean the model does not hallucinate but this is another story.

param_gupta

Pretty interesting how a simple example like this makes the idea so easy to understand.

revexos

Startup coming out of 2 years of stealth to be reproduced this easily

jorisw

Highly suspect of content marketing. Ends with referring to a product, and saying "this is a parody post", after pretending to make a serious point.

baobabKoodaa

I'm so sick of seeing these people who "made Jev in 25 lines of Python" or whatever the flavor of the day is. Do you people seriously think that Qwen3-0.6B-Q8_0.gguf is frontier intelligence? If you want to argue that Jev is NOT frontier intelligence, then go make that argument. Don't try to pretend that Qwen3-0.6B-Q8_0.gguf is frontier intelligence. That's retarded.

K0IN

a hile ago (when big providers still provided logprobs) i created a VS Code highlighter that visualizes unsure tokens. Since most chat models want to answer with a human-readable message i think their logprobs are not as meaningful. It would be interesting to see if one choice is like "correct" and if the model wants to choose it more often, cause it might not answer the question but to prose to the user.