We stopped AI bot spam in our GitHub repo using Git's –author flag

ildari

Hi HN community, I wanted to share our approach to reduce amount of AI slop PR's and issues in our repo. We enabled "require prior contribution" flag on GH and created a CI script that creates a tiny commit co-authored with you, if you pass captcha on our website. Worked really well and we were able to block at least 500 bots in the first week. Sharing a screenshot from cloudflare: https://archestra.ai/hn-comment-cloudflare-challenge-outcome...

silverwind

PR spam is a major problems for repo that run bounties. Maybe GitHub should temporarily block accounts from raising PRs if like 95%+ of them are getting rejected.

zer0tonin

> Should we stop giving fun test tasks to our job candidates? Yes

hiccuphippo

The irony of the .ai domain.

ramon156

See, this is an article that uses dashes correctly. It adds value, creates a bit of buildup

arecsu

Makes me wonder if an ELO-based system would work to mitigate these issues. People who merged PR successfully onto a project, that had real issues acknowledged, the quality of their responses measured by other users reactions or something, etc, multiplied possibly by the degree of importance of the project where their activity has been made. Won't be about human vs AI, but actual helpful effective being vs low effort/spammy contributions. Issues and PRs could be sorted and filtered by their ELO score. I'm saying ELO as analogy to "score based given the context", not really a 1:1 translation of the ELO system. Negative score would be reports from other users because of spammy content or not acknowledged issues, with a middle ground of neutral score (+-0) or little positive score to issues or whatever with clear good intention, but couldn't reach a proper merged PR or were not issues (e.g. issue existed but wasn't the correct repo to be addressed, PR was good but needed other stuff to be implemented prior to it, maybe in the long run, etc)

petterroea

What I see is a (clever) hack, and GitHub continuing to provide good tools to its users.

captn3m0

This has a security implication which is overlooked. Contributors to a repository have higher rights, such as avoiding approval requirements for fork PR runs. GitHub warns in the docs: > When requiring approvals only for first-time contributors (the first two settings), a user that has had any commit or pull request merged into the repository will not require approval. A malicious user could meet this requirement by getting a simple typo or other innocuous change accepted by a maintainer, either as part of a pull request they have authored or as part of another user's pull request.

_joel

Woudln't it be trivial to farm the stats needed to pass the bot checker's theshold?

zzzeek

so...they are manually re-setting the "interaction limits" over and over again, since they are only temporary? why not use hooks to automatically reject issue comments / PRs etc. from users that didnt go through onboarding, rather than repurposing GH features that aren't really designed for that use (and are hence in danger of being changed someday)?

IshKebab

That's a neat way to interface with GitHub's authentication system, but I don't see how they've solved the fundamental problem because their whitelisting process is just "click ok fine 10 times". Why won't the slop peddlers just do that too?

optionalsquid

I don't have a better solution, unfortunately, but it doesn't seem seem to like the spam problem has been solved. It has just been moved from pull requests to commits: Currently, more than 10% of all commits in the archestra repo are essentially noise (369 of 3521 commits), accounting for more than half of all commits in the last month (303 of 578 commits). But maybe (probably) the amount of such commits will go down over time, compared to the growing amounts of AI slop

Muromec

How is the status revoked without rewriting git history?

aizk

I'm not sure why gh hasn't already implemented stricter measures / filters / tools for PRs. It would cut down on spam and also help save their servers that can't handle the increased AI load!

infinitifall

Is the solution to everything simply more catgirls [1]? Proof-of-work was, after all, about countering email spam. PR spam is but the latest in that long tradition. 1- https://anubis.techaro.lol

thih9

> It's not a contract job— it's our optional way of saying thank you to the community. The writing style in their onboarding doc has common AI tells (in the quote: em dashes, “it’s not A, it’s B” sentence). I can understand that, perhaps they want to fight fire with fire or don’t have time as they already say. Still, it all feels like inadequate half measures to me.

jart

This is great example of the toxic effect money has on open source. Reward people with respect and recognition instead. Weird anonymous accounts no one's ever heard of will leave, because someone (or something) who's concealing their identity has nothing to gain from recognition. Honestly GitHub should have a real names policy. Because if you're not Satoshi Nakamoto then there's only three reasons I can think of to be anonymous on GitHub: (1) to avoid obtaining your employer's authorization, (2) to spam, harass, and engage in toxic behaviors, or (3) you're not even human. All three of these are the last things I want when engaging on the GitHub platform. Don't get me wrong, I love robots. But I'm perfectly capable of talking to the robot on my own. I don't want to talk to your robot. I also don't want people slipping me intellectual property below the board without their employer's consent. And I certainly don't enjoy all the hate and harassment. GitHub has tried to help with the last part, by making overt displays of hate something that can get you in trouble. The issue is that people just get more guilesome with more anonymous accounts, because the issue was never disrespect (which can actually be strategic and pro-social if we look at Torvalds' career), but rather bad faith participation. If GitHub can guarantee that all its users are human real names good faith actors, then we might be able to start talking about open bounties.

embedding-shape

Sounds kind of weird that the blog post complains about `poisoning the conversation with pointless "implementation plans"` when literally they ask for that, after attaching $900 USD bounty to a very under-specified issue, and even replies with "Do you have an implementation plan in mind?" to some of the first "attempters". Sounds like they got exactly what they'd been asking for, and even before LLMs if you pulled something similar, the effects would have been similar.

rglullis

'I will take "problems that could be easily be solved by implementing a Pfand system" for $200, Alex.' Seriously. Just ask for a US$10 deposit for the each PR. If the PR is accepted (not even merged, just accepted as "this is a good effort"), give it back. Hell, give double the amount for good effort and you got yourself a cheap way to attract good contributors. Best case, bots will balk at the payment. Worst case, the funds can be used to hire someone specifically for triage.