LLMs.txt For AI SEO: Is It A Gain Or A Waste Of Time?

LLMs.txt claims AI SEO gains, but is it really delivering? Discover the truth about the potential benefits, drawbacks, and the most effective techniques.

The debate surrounding LLMs.txt, a file proposed to help guide AI search engine visibility, has sparked discussion and confusion. Although many SEO platforms and tools promote LLMs.txt as an essential feature, the truth is that it’s merely a suggestion, and no AI engines have yet adopted it. 

Confusion Sparked by Third-Party SEO Tools

Google’s John Mueller recently addressed misconceptions about LLMs in a Reddit discussion. A SEO user who was worried about a Semrush audit that flagged an LLMs.txt file as insufficient demanded that the file be fixed with a “404” error.

The question was:

“Why is SEMRush showing that the /llm.txt is a 404? Yes, I. know I don’t have one for the website, but, I’ve heard it’s useless and not needed. Is that true?

If i need it, how do i build it?

Thanks”

Semrush’s documentation suggests that the absence of an LLMs.txt file poses a risk of misrepresentation to AI systems, advising SEOs to be aware of issues that might compromise AI results. But John Mueller confirms the file is not needed, advising against putting effort into untested strategies:

“If your site lacks a clear llms.txt file it risks being misrepresented by AI systems.

…This new check makes it easy to quickly identify any issues that may limit your exposure in AI search results.”

LLMs.txt Is Not Necessary

The documentation suggests it is a ‘risk’ not to have an LLMs.txt, but in reality, there is no risk, as no AI platform actually uses it. That is likely why the Redditor asked, ‘If I need it, how do I build it?’

Google’s John Mueller has confirmed that LLMs.txt is not necessary.

He explained:

“Good catch! Especially in SEO, it’s important to catch misleading & bad information early, before you invest time into doing something unnecessary. Question everything.”

Why AI Platforms Avoid LLMs.txt

None of the important AI platforms has officially adopted LLMs.txt and its markdown (.md) files. One of the main reasons is the trustworthiness. 

Content on the page is easily transparent and readable for both AI and users. However, the markdown or text file may be altered by unscrupulous SEOs to trick AI systems.

A research paper from 2024 on SEO Adversarial in large Language Models demonstrated:

“…an attacker can trick an LLM into promoting their content over competitors. Preference Manipulation Attacks are a new threat that combines elements from prompt injection attacks… Search Engine Optimization (SEO)… and LLM ‘persuasion.’

We demonstrate the effectiveness of Preference Manipulation Attacks on production LLM search engines (Bing and Perplexity) and plugin APIs (for GPT-4 and Claude). Our attacks are black-box, stealthy, and reliably manipulate the LLM to promote the attacker’s content. For example, when asking Bing to search for a camera to recommend, a Preference Manipulation Attack makes the targeted camera 2.5× more likely to be recommended by the LLM.”

These tactics could unfairly boost popular content, highlighting the danger in creating exploitable AI-specific files. Therefore, relying on web content that is visible to the user is safer and more secure than LLMs.txt files.

Views by SEO Plugins

SEO tool providers differ in their opinions on LLMs.txt:

• Squirrly included LLMs.txt support after users asked for it, but the company clearly states that it will not increase AI Search visibility. They stress the importance of transparency and write:

“I know that many of you love using Squirrly SEO and want to keep using it. Which is why you’ve asked us to bring this feature.

So we brought it.

But, because I care about you:

– know that LLMs txt will not help you magically appear in AI search. There is currently zero proof that it helps with being promoted by AI search engines.”

• Rank Math presents LLMs.txt as a curated source for AI chatbots to utilize when analyzing content, which improves the chances of accurate citations. However, this is a misrepresentation of actual AI behavior, as AI bots typically rely on standard HTML content rather than curated Markdown files. They write:

“So when an AI chatbot tries to summarize or answer questions based on your site, it doesn’t guess—it refers to the curated version you’ve given it. This increases your chances of being cited properly, represented accurately, and discovered by users in AI-powered results.”

• Yoast SEO offers an intermediate position, explaining LLMs.txt’s potential uses without exaggeration and with cautious words like “can” and “could.”

The Self-Reinforcing Misinformation Loop

The growing concern over AI transparency is prompting many SEOs and business owners to utilize LLMs.txt, believing it to be essential. SEO tool makers respond by providing LLMs.txt features, which inadvertently reinforce the belief that the files are required.

This loop continues, although there is no AI search engine currently supporting LLMs.txt, which makes it, at best, a speculative approach rather than a practical and common one.

Bottom Line

LLMs.txt is an intriguing idea, but it is not yet an evidence-based benefit in SEO using AI. Marketing professionals and SEOs should focus on creating high-quality, clear, and transparent content on the web, rather than seeking untested technological add-ons.