AI Search Redirects Users to 404 Pages Nearly 3X More Than Google

AI search leads to 404 pages three times more than Google, pointing out potential issues in the AI search experience and the consequences of user navigation.

AI search engines, and ChatGPT in particular, return broken links nearly three times more often than Google’s search engine, according to an analysis of 16 million URLs conducted by researchers at Brown University.

ChatGPT Leads in Hallucinated Link Rates

1% of URLs clicked from ChatGPT result in a 404 error page vs. 0.15% on Google, according to an Ahrefs investigation. Taking into account all the URLs referenced by ChatGPT and not just the ones that were clicked on, the broken link rate jumps to 2.38%, compared with 0.84% for the top Google search results.

For broken URLs users clicked through†, other AI-assistants rated as follows:

• Claude: 0.58%
• Copilot: 0.34%
• Perplexity: 0.31%
• Gemini: 0.21%
• Mistral: 0.12% (the lowest broken link rate and generates the least website traffic)

AI Generates Fake URLs

The two key reasons are revealed in the study:

• AI pulling from old data where the URL used to be, but was taken down or moved.
• AI is dreaming up plausible but nonexistent URLs with some relation to certain sites.

Ryan Law, from Ahrefs, shared that AI-built fake URLs look “normal,” like “/blog/internal-links/” and “/blog/newsletter/” and are following site structures, but in reality they do not exist on Ahrefs.

Limited Traffic Impact So Far

Although these broken links are so widespread, their impact on site traffic is fairly minimal. AI assistants account for about 0.25% of web traffic, and Google is responsible for 39.35%. Their phony URLs now make up a minuscule portion of an already marginal source of traffic.

But as researchers use more AI, this problem could get worse. The study also found that 74% of the new web pages it identified have AI-generated text, which may even include fake links that search engines could index, further spreading the problem.

John Mueller’s Prediction

Google John Mueller sounded the early alarm on the rising “hallucinated links” in March, and predicted:

“slight uptick of these hallucinated links being clicked”

Mueller predicted this over the next 6 to 12 months and recommended not aggressively chasing accidental traffic and to make 404-page experiences better, which makes a good amount of sense in light of the relatively inconsequential current effect, as reported by Ahrefs.

Mueller also argued the problem would lessen as AI systems became more effective at interpreting URLs.

Moving Forward

In the meantime, the best practice for websites is:

• Making good 404 pages that help out users who hit a broken link
• Applying redirects to dummy URLs that at least generate some traffic

This tempered approach addresses the problem without overreacting to what, for most sites, remains a relatively minor issue.

Final Thoughts

Such considerations reiterate the development of AI search and emphasize the need to balance innovation and quality and reliability in digital content discovery.