Here's a story you should sit with for a minute before you get back to your content calendar. An SEO wanted to know how easily a made-up fact becomes an AI-repeated fact. So she made one up.
The experiment
Lily Ray published the account on her Substack under the title "The AI Slop Loop." The setup was simple and a little mischievous: invent a Google algorithm update that never happened, a "September 2025 Perspectives Core Update," and seed it into the ecosystem through a handful of AI-generated blog posts, the kind of low-effort, high-volume content that already floods every SEO news cycle. Then wait and watch what the AI answer engines do with it.
She didn't have to wait long. Within 24 hours, AI Overviews and Perplexity were both citing the fake update as if it were settled fact, repeating details from her own seeded posts back to searchers as established history. Ray, with the kind of dry humor that made the post spread, joked that Google had apparently approved the whole thing "between slices of leftover pizza." The joke was the point: nobody approved anything. The loop approved itself.
What makes this land harder than a typical misinformation cautionary tale is who ran it. Ray isn't a random content farm operator stumbling into an exploit. She's one of the most credentialed analysts in the SEO industry, someone who spends her working life reading Google's actual announcements closely enough to spot the difference between a real update and vapor. If someone with that level of pattern recognition can manufacture a convincing fake in an afternoon, the bar for doing it accidentally, or maliciously, at scale is lower than most brands assume.
How fast the loop closed
It's worth naming why this particular experiment worked so well as a demonstration, beyond the humor of it. SEO news is a genuinely fast-moving beat with real information asymmetry: most people, including plenty of practitioners, can't personally verify whether a named update actually rolled out, so they rely on secondhand summaries. That's precisely the kind of information gap a fabricated but confidently-written claim exploits best, and it's exactly the gap AI-generated content mills are built to fill at volume, whether or not anyone's deliberately gaming it the way Ray was.
Twenty-four hours is the number that should stick with you. That's not a slow leak of misinformation working its way up through months of backlinks and republishing, the way a bad fact used to spread. That's a same-day round trip: fabricate a claim, publish it in a form an AI-generated content mill would produce anyway, and have it repeated as truth by two of the biggest answer engines on the internet before the next news cycle even starts.
“The loop didn't need a conspiracy. It needed content that looked confident, published in a format the model already trusted, and no one on the other end checking whether the update actually happened.”
Why this isn't just a funny story
It would be easy to file this under "amusing SEO prank" and move on. That undersells it. Ray's experiment is a small, controlled version of something that happens constantly at scale, without anyone deliberately engineering it: a claim gets published somewhere with enough surface confidence, an engine retrieves it because it's structurally easy to lift, and the claim becomes the answer, corroboration or not. The signals that get you cited, extractable structure especially, don't distinguish between a true claim and a confidently formatted false one. That's not a flaw anyone's rushing to fix, because fixing it is genuinely hard.
Consider what this means for the ordinary, non-mischievous version of the same mechanism. A competitor doesn't need to run a deliberate disinformation campaign to get an inflated claim about their product cited over yours. They just need to publish it somewhere structurally easy to lift, with enough surface confidence, before anyone corrects it. Ray proved the loop closes in about a day when someone is actively trying to trigger it. There's no reason to assume it moves meaningfully slower when nobody's trying, and everyone assumes someone else is watching.
The bigger accuracy problem
Ray's stunt lines up with harder research on the same weakness. NYT-affiliated research into AI Overviews found the feature accurate 91% of the time, which sounds reassuring until you read the second number: 56% of those correct responses were still ungrounded, meaning the citation attached to the correct answer was bad, irrelevant, or didn't actually support the claim being made. The engine got the right answer and cited the wrong reason for it more than half the time. That's the accuracy problem underneath the accuracy statistic.
| METRIC | FINDING |
|---|---|
| AI Overview response accuracy | 91% (NYT-affiliated research) |
| Correct responses with ungrounded citations | 56% |
| Time for a fabricated claim to be cited as fact | 24 hours (Ray's experiment) |
| GPT-5.4 false-claim rate vs GPT-5.2 | 33% lower (model-level improvement) |
The model-level trend is moving the right direction; OpenAI's GPT-5.4 reportedly produces 33% fewer false claims than GPT-5.2 did. But that's a slow structural fix arriving underneath a system that's already citing things at Ray's 24-hour speed. The gap between "the model is getting more careful" and "the loop still closes in a day" is exactly where a bad fact, or a competitor's bad fact about you, lives right now.
What it means for your brand
This connects to a second, related piece of Ray's research, covered by Search Engine Land: across 100 B2B software queries, Google's AI Overviews triggered on 80 of them and generated 323 citations of brands' own self-serving "best of" listicles. In 224 of those cases, about 69%, Google cited the brand's own page but didn't actually recommend that brand, pulling the format and citing the source while landing on a different verdict. Some SaaS companies lost 30 to 50% of their category visibility leaning on exactly this tactic. Third-party sites, Reddit, Forbes, YouTube among them, won most of the actual recommendations instead.
Read those two studies together and a pattern emerges that's bigger than either one alone. In the fake-update experiment, the engine cited a source with zero real corroboration because the content was structurally confident. In the self-serving-listicle research, the engine cited a real, legitimate brand page and then ignored its verdict anyway, in 69% of cases, because the model had its own read on which answer was actually credible. Both point to the same underlying reality: citation and endorsement are not the same event, and confusing the two is how a brand ends up cited constantly while losing every recommendation that matters.
There's a temptation to read all of this as a reason to distrust AI answer engines wholesale, and that's the wrong lesson too. A 91% accuracy rate, badly grounded citations aside, is not a system that's fundamentally broken. It's a system that's good enough to be trusted by default and imprecise enough, at the sourcing layer specifically, to reward anyone willing to game it. Those two things being true at once is exactly what makes this a marketing and monitoring problem rather than a reason to write the whole channel off.
How to not be the next fake fact
There's a broader implication here for anyone building a content or PR calendar around AI visibility: speed of correction is now a competitive asset in its own right, not just a customer-service nicety. A brand that can get a wrong or unflattering claim corrected within a day is operating on the same timeline the misinformation itself moves on. A brand that routes corrections through a quarterly content review, or worse, doesn't monitor at all, is playing a different, much slower game against a mechanism that doesn't wait for anyone's editorial calendar.
Run your own version of Ray's check, minus the fabrication: search your category's core questions across ChatGPT, Perplexity, and AI Mode weekly, and read what's actually being cited, not just whether you show up. If you find a claim about your product that's wrong, ungrounded, or lifted from a low-effort AI-slop post, treat it the way you'd treat a factual error in a major publication, because functionally, that's what it now is. That's the same monitoring discipline we build into our content marketing engagements and GEO work: weekly citation checks, not quarterly reports, because a wrong fact about you can close its own loop faster than your reporting cycle catches it. We rebuilt this monitoring cadence for a healthcare technology client after a competitor's inflated claim showed up, uncorroborated, in three separate AI answers, the same blended-visibility blind spot we've written about in why your mention-rate dashboard is lying to you. The tools that got it cited took a day. Getting it corrected took considerably longer, which is exactly why you check before you need to.
See where you are cited today
A free snapshot audit of your rankings and AI citations before we ever talk.
Josh leads work at the intersection of SEO and generative engines at Something Inc., helping B2B brands get ranked and cited across every major AI engine.