We are retiring our bug bounty program

k2xl

Isn't there some alternative approach? I.e when someone submit ai slop they get a strike. Three strikes and you are suspended from submitting to the bug bounty for x months/years? *Edit - I get it. It seems like the authentication is a challenge.

ToucanLoucan

Oh look it's more of exactly what AI skeptics said would happen: low effort bullshit generated at scale making life hell for people actually trying to make things. That's wild. Edit: it is genuinely wild, I don't know of another product category that selects so perfectly for the WORST type of person to be it's enthusiast. Just every single person I see hyped about AI is fucking insufferable on at least one and usually multiple axis.

wg0

Which goes on to prove that bottleneck isn't in writing the code. It is in reading and understanding the code. We all had that one "productive" engineer in our teams who would write huge PRs that would have large swaths of refactoring whether warranted or not and that was way before anyone even could imagine in their wildest dreams that neural networks could generate that huge amounts of code. The net effect of such a "productive" engineer always was that instead of increasing the team velocity, team would come to a crawling pace because either his PR had to be reviewed in detail eating up all the time and/or if you just did cursory LGTM then they blew up in production meanwhile forcing everyone back to the drawing board but project architecture would have shifted so rapidly due to his "productivity" that no one had a clear picture of the codebase such as what's where except that one "super smart talented productive loyal to the company goals" guy.

mikemarsh

An interesting "conundrum" (at least from my outsider perspective): how many of those bot requests are from agents that utilize Turso on their backends?

jmuguy

I wonder what Hacktoberfest would look like now if they were still giving out t-shirts to everyone. Probably not enough cotton in the world. It can't be on individual maintainers to stop this, imo its on Github (and Gitlab) to stop these sort of accounts from even getting to the point of submitting PRs. Its essentially spam. Look at the user who created the first PR they reference https://github.com/Samuelsills . This is not an account that should be allowed to do anything close to opening a PR against a well known repo.

MostlyStable

Closing the program is totally reasonable. However, there is another option: Make submitters pay a nominal fee that is returned in the case that a real bug is found.

phyzix5761

Can't they just beat them at their own game and deploy their own AI bots to pre-screen the PRs?

satvikpendem

Has anyone used Turso in production? It's an SQLite compatible rewrite in Rust but with added features like multiple writer support and being open to external contributions which SQLite is not. I was thinking of using it for my full stack Rust apps just so everything works with cargo and I don't have to bring in SQLite separately.

Lalabadie

Good time to mention this fantastic repo acting as a bot honeypot: https://github.com/UnsafeLabs/Bounty-Hunters The corresponding leaderboard: https://clankers-leaderboard.pages.dev

Havoc

Definitely feels like we're heading towards an eternal september (or already arrived). ...large swaths of approaches on online engagement just becoming non-viable

pscanf

We sorely need a way to reliably detect AI slop, but unfortunately it doesn't seem possible and it's just getting harder and harder. Last month I tried my hand at finding a way to tell whether an OSS project is slop or not, based on the amount of "human attention" it received vs the amount of code it contains. The idea is that a 100k LOC project which received 3 days' worth of attention from a human is most certainly slop. The approach doesn't work very well, though¹, mostly because it's hard to gauge the amount of attention that was given. If I see one commit with +3000 LOC, I can assume it's AI-generated, but maybe you're just the type of dev that commits infrequently. Maybe we need some sort of "proof of human attention" for digital artifacts, that guarantees that a human spent X time working on it. ¹ I wrote about it here https://pscanf.com/s/352/

curtisblaine

Bots are using real tokens for this. So, ultimate honeypot idea: post heavily commented skeleton code in a github repo, promise a generous money reward for closing issues and never pay anyone. See the bots swarm and burn their tokens to write code for you.

singpolyma3

It's a bit odd that this comes today after so many other projects reverse this finding.

overgard

The weird thing is it can't be that economically feasible to burn a ton of tokens in the hopes that you might get a bounty.. seems like a great way to set money on fire.

adamtaylor_13

Being a verifiable human identity (not as-in age verification or whatever) but as in having a known, public, reputation online will go a long way in this new slop-first world.

bee_rider

Possibly stupid question (this is outside my wheelhouse): is there any way a final full run of the simulator test cases (presumably required to make sure the submitted simulator changes don’t break the thing) could act as a proof-of-work?

arian_

we automated finding bugs. then we automated submitting bugs. now we're automating rejecting submissions. at no point did anyone automate fixing the bugs.

AlienRobot

I'm sorry but I find the slop PR's hilarious. >the author just injected garbage bytes manually into the database header, and then argued that this corrupted the database >Steps to reproduce: Modified cli/main.rs to include a Vec with limited capacity. Forced a volatile write beyond the allocated bounds using std::ptr::write_volatile. >author claims to have found a critical vulnerability that allows for the execution of arbitrary SQL statements. Imagine that? A SQL database that allows the execution of SQL statements. How can we ever recover from this. I wonder why are they even doing this. Do any of these PRs ever win any money? It feels like they are burning down a forest thinking they'll find gold if they do it, without any evidence that there will be any gold after the forest is burnt down.

jrgd

it seems we all will slowly learn to live within new contexts; i really appreciate their openness about it and it gives me insights to munch on thanks to you all also to ring in with dev-style annecdotes (i'm stilllearning everyday, and hope to continue for a long time): those big-prs and tactical tornadoes stories are helping keep the crafts and thinking afloat, somehow.

andai

I don't get it. Can't they ask Claude to check slop? This sounds like a bit of a baby/bathwater situation. (Okay Claude is too expensive, but Deepseek can probably handle it.)