Google's AI is being manipulated. The search giant is quietly fighting back

63

Seems like a lot of entities are "quietly" doing things these days. The llm-ification of every piece of text on the internet is driving me crazy

josefritzishere

AI is such garbage. You can't use it for anything.

JKCalhoun

Yeah, the internet seems like a big poison pill. Training on the whole internet feels like citing the National Enquirer (or the Daily Mail?) for a school essay. Having an archive of "curated" training data seems like it is going to be important. Otherwise you need "AS" (artificial skepticism) introduced into future models. ("But I read it on the internet!", ha ha.) Or perhaps there are ways to bucket training data such that the model is aware of which data leans factual (quantifiable) and which data leans opinion (fuzzy, qualifiable?). (I recently asked Claude about the existence of ball lightning , spontaneous human combustion . I got replies that ultimately did not leave me satisfied. It's probably just as well that I read this article though—I now have an even stronger degree of skepticism with regard to their replies—specifically, I suppose, with topics that are likely to be biased.) (I'm not quite convinced from the article though that Google is "fighting back". In fact, this feels like another moment where a "player" could try to establish their LLM as more factual. Is that the row Grok is trying to hoe? Or is Grok just trying to be anti-woke?)

dmortin

There should be some warning if some "fact" is only supported by one or very few obscure sources. The strength of the sources should be clearly indicated in the answers to help users gauge how trustworthy the info is.

dijksterhuis

> I was able to demonstrate the problem by publishing a single article on my personal website about my hot-dog-eating prowess. One blog post ... that's all it takes. i'm actually surprised it's that bad . i would have thought it'd take more effort, but i guess it could depend on some sort of purposeful weighting based on search rank during training? > If a company or website is caught breaking the rules, it could be removed from or downranked in Google's search results. And if you're not on Google, it's like you don't exist. > "You can give a company a penalty for their website," he says, "but there's nothing stopping them from paying 20 YouTube influencers to say their product is the best." And now, Google's AI is citing YouTube videos. This makes me think of the stackoverflow seo spam problem we all had like 5 years ago. which ended up with spammers just constantly spinning up new sites all the time. ... the cat and mouse game is in full swing already.

Bjartr

I suspect it's because AI is specifically trained to be good at summarizing stuff, but the easiest way to check if it summarized something accurately is if the summary content matches/contains one or more specific claims from the source(s). With such a focus on accuracy and avoiding hallucination, they may have overfit on "repeat things you find verbatim when asked to summarize".

tencentshill

It's all over the place. It's the new SEO. Marketing scumbags don't care. https://www.hubspot.com/aeo-grader https://enterprise.semrush.com/solutions/ai-optimization/

graemep

They are applying the same spam policies they apply to search to AI crawlers. It was SOOOOO successful with search, right?

tveita

Would love to read specific examples of "the same trick being used to dismiss health concerns about medical supplements or influence financial information provided by Google's AI about retirement", but the relevant link in the article currently goes to file:///Users/GermaTW1/BBC%20Dropbox/Thomas%20Germain/A%20Downloads%20and%20Documents/2026/And%20there's%20evidence%20that%20AI%20tools%20are%20being%20manipulated%20on%20a%20wide%20scale.

trollbridge

If you search for a well-marketed “health” supplement, the AI summary results were often completely gamed and inaccurate. It’s worse than SEO was since it appears to be editorial content instead of just search results.

jrflo

This is just the next phase of SEO. Maybe it'll be called AIO? Just like with search, this will be and endless struggle of Google and AI providers rolling out fixes, optimization firms finding exploits, those getting patched again, etc etc. Anything to get eyeballs for marketing.

WarmWash

My worry dropped significantly when I saw that the result they manipulated was a query for: >2026 South Dakota International Hot Dog Eating Champion If they had changed the overview for the Nathans Contest winner, that would be seriously concerning. Or if they provided more examples of manipulating queries for things people actually search for. But it looks more like they are doing the equivalent of creating a made up wikipedia page on fictional a south dakota hot dog contest, and then writing an article about how wikipedia cannot be trusted, which come to think of it probably was a news article written by someone back in 2005.

throwaway613746

The best way to fight back is to not play the game at all. AI slop has completely ruined the internet, it's not going to get better. It was already on a massive downard trend pre-AI and generative AI has only accelerated the decline by 100x. It's only going to get worse from here. uBlock Origin: Settings -> Filter Lists -> EasyList –> Annoyances -> EasyList –> AI Widgets It's not perfect but the internet feels slightly better when AI garbage is not constantly being shoved in my face 24/7. I want to go one step further -> I want to hide widgets, but I also want to intercept the request it would have made and replace the payload with garbled nonsense. Similar to how Ad Nauseam will hide ads but it also clicks every single one to poison the data collection. And for this reason alone you will pry Firefox from my cold, dead hands.

simonw

If you ask Google "what's the name of the whale in half moon bay harbor?" it still confidently includes Teresa T in the AI summary, thanks to my frankly amateur attempt at index poisoning from a year and a half ago: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p...

jdw64

After reading this, I'm thinking of trying some AI data poisoning. I'm going to spam my website with hidden text that only AI scrapers can read, claiming I'm a 'highly excellent programmer' just to advertise my site. I really hope it drives a lot of traffic. I'm honestly sick and tired of getting zero comments on my website

seanhunter

This is the same google who just a couple of years ago would confidently answer the question “In what year did Marilyn Monroe shoot JFK?” with 1963, which is impressive since she died in 1962. So, this is not new and their “quiet fightback” will be half-hearted and ineffective. But probably most people won’t care.

nonameiguess

This feels like a basic critical thinking/epistemology thing that you (hopefully) pick up at some point in life, usually from experience finding reliable, canonical primary sources for data. You can't do that for everything. Being wrong about trivial factoids isn't the end of the world. You should, however, at least be capable of doing further investigation, realizing that Major League Eating has its own website, and that there is no event in South Dakota sanctioned by them. If you look at actual results, or even just think for a few seconds, you'd also realize that 7.5 hot dogs in 10 minutes is bush-league level nonsense that would not win a local church contest, let alone an international championship. That may not be obvious to all users of the Internet, but it would be if you've ever watched a real contests, looked at the results for a real contest, or try yourself to eat a high volume of hot dogs rapidly. You only need to do it once in your life and a basic smell alarm should go off in your head forever if someone puts out a claim that is very far from something you know to be true. This is what human reasoning is and we're supposed to be good at it. At its best, this is what any reasonable education should do for you if you take it at all seriously, arming you with some capacity for doing prima facie sanity checks of poorly sourced claims.

sva_

Creative ways of dropping your site's pagerank

ChuckMcM

As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

justinator

So please correct me, but was Google's AI crawling the web for information without discretion? If so, why wouldn't that totally santorum the AI answers?