Pangram – AI detector for text and images

gleezard

How accurate is this? I want to see a negative before i provide you with a positive.

ElProlactin

> An AI detector that actually works. I detect Claude.

BobbyTables2

I detect SPAM.

jdc-pub

I’m surprised at the current sentiment in the comments. Pangram is amazing and has really interesting engineering too. I would have guessed that reliably identifying LLM generated text was not possible without watermarks.

firefoxd

People expect a binary response, is it AI generated yes or no. But it's more complicated than that. For example, if you see an emdash, it's probably AI generated. But it can also mean the author used it for fixing grammar or tenses. LLMs can't help but try to help. The same for it's not X, but Y. Sure it's a known pattern, but it's not like people don't use this trope all the time. In my experience, Pangram is great for detecting an author who is trying to pass someone else's work as theirs, or if they are tackling a subject they have little to no knowledge in.

aizk

I don't work for Pangram, but I do a lot of writing at work. API docs, blogs, code, tweets, linkedin, all that. If you've tried AI detectors a couple years ago, they're basically in the same position that coding agents were a few years ago, where everyone was skeptical at first, but the tech has gotten a lot better. Give it a shot, it's quite good. They are slightly tuned a bit towards classifying things as AI, but I imagine that's deliberate. The only thing is that their models are pricey, but, very useful.

kqr

I didn't want to shell out $20 a month for the general thing, so I spent $90 on data collection and built my own for code comments specifically. It runs locally in your browser with a relatively small classification model trained on old-school stylometric features. You can try that before turning to Pangram for uncertain cases, if you wish.[1] It's easy to get high accuracy numbers if you're testing on long (50+ words) texts. Much harder when the documens are short, as code comments tend to be.[2] [1]: https://xkqr.org/aicomment [2]: https://entropicthoughts.com/better-ai-comment-classifier

mdspan

Based on their methodology it looks like the accuracy figures (99.82% for Opus 5) are the true positive rate rather than a combined metric that factors in the false positive rate as well. They claim 1 in 10,000 but it would be nice if we had a per-model breakdown for that specific test.

ShinyLeftPad

First they sell a problem, then they sell a UGC honeypot masquerading as solution to that problem.