LLMs.txt

AI & Generative Search

Also: llms.txt · LLM Text File

What it isA file that guides AI models to your best content

MaturityProposed standard, not adopted

Reality checkNo major AI vendor honours it yet

Lives atyoursite.com/llms.txt

Quick definition

LLMs.txt is a proposed text file you place at the root of your website to help large language models (the AI behind tools like ChatGPT) find and understand your most important content. It works like a curated map for AI, pointing to clean versions of your key pages.

What it actually means

LLMs.txt is a proposal, not a rule. Think of it like leaving a welcome note and a floor plan at the front desk for AI systems that visit your site. The idea is that a large language model, rather than wading through your bloated HTML and navigation menus, reads a tidy markdown file that says 'here is what matters and here is where the clean version lives'.

The concept borrows its logic from robots.txt, the long-standing file that tells search crawlers where they can and can't go. But there's a crucial difference. Robots.txt is honoured by every serious crawler. LLMs.txt is honoured by almost nobody.

At time of writing, no major AI vendor has committed to reading it during answer generation. OpenAI, Google and Anthropic have not adopted it as a signal. So the file sits on your server as a well-intentioned gesture that the systems it's aimed at are mostly ignoring.

That doesn't make it worthless. Standards start as proposals that get traction. But you should treat LLMs.txt today as a cheap, low-effort experiment in generative engine optimisation, not as a lever that moves anything measurable. The honest position is: harmless to add, premature to rely on.

LLMs.txt is a polite suggestion nobody important has agreed to read yet.

How it shows up

LLMs.txt shows up as a single markdown file at yoursite.com/llms.txt. Inside it, you list your key pages with short descriptions and links, often pointing to plain-text or markdown versions of each page. Some sites add a longer llms-full.txt that inlines the full content for models to read directly.

Where it does not show up is anywhere measurable. There's no report in Google Analytics 4, no field in Search Console, no way to confirm an AI model read it. Unlike a UTM parameter that shows you the click or a conversion event you can count, LLMs.txt gives you no feedback loop. You're publishing into a void and hoping the standard gains traction. That uncertainty is the whole story of the term right now.

The Australian context

Australian search volume for LLMs.txt is effectively zero today. Almost nobody is looking it up, which tells you how early this is. The businesses experimenting with it here are mostly tech-forward agencies and SaaS companies watching the AI search shift closely.

That early stage is actually the argument for a low-cost look now. If the standard does get adopted, being among the first Australian sites to implement it cleanly costs almost nothing. But temper the enthusiasm. Australia's smaller market means local vendors rarely drive standards adoption. We'll follow whatever OpenAI and Google decide, and neither has decided anything yet.

Where people get this wrong

Treating LLMs.txt as a ranking factor for AI search.No major AI system has confirmed it reads the file during answer generation, so there is no ranking effect to claim. It's a proposed convention, not an active signal.

Confusing LLMs.txt with robots.txt as a way to block AI crawlers.LLMs.txt guides AI toward content. Blocking AI crawlers is done through robots.txt directives and server rules. They solve opposite problems and using one for the other does nothing.

Prioritising LLMs.txt over clean content and schema markup.The things AI systems actually use today are well-structured content and structured data. Spending effort on an unadopted file while those basics are broken is backwards.

LLMs.txt vs Robots.txt

	LLMs.txt	Robots.txt
Purpose	Guides AI models toward your best content	Tells crawlers where they can and can't go
Adoption	Proposed, honoured by almost nobody	Universal, honoured by every serious crawler
Direction	Invites AI in	Can block crawlers out
Measurable effect today	None you can confirm	Directly changes what gets crawled

Common questions

Do I need an LLMs.txt file?

Not urgently. No major AI vendor honours it yet, so it changes nothing measurable today. If your content and schema markup are already solid, adding the file is a cheap experiment worth an hour. If they're not, fix those first. LLMs.txt is a nice-to-have, not a priority.

Is LLMs.txt the same as robots.txt?

No. Robots.txt is a universally honoured file that tells crawlers where they can go and can block them out. LLMs.txt is a proposed file that invites AI models toward your best content. Opposite intentions, and only robots.txt actually works today.

Does LLMs.txt help me appear in ChatGPT or AI Overviews?

There's no confirmed evidence it does. OpenAI, Google and Anthropic have not adopted it as a signal for answer generation. What genuinely influences how AI describes you is clear content, structured data and general authority. Treat LLMs.txt as a bet on a future standard, not a working tactic.

How do I create an LLMs.txt file?

Write a markdown file listing your most important pages with short descriptions and links, then upload it to your site root so it lives at yoursite.com/llms.txt. Some sites add a fuller llms-full.txt with inlined content. It's a low-effort task, which suits its low-certainty payoff.

Debrief

Get the next one

No spam. No fluff. Just the next article, straight to your inbox.

Keep exploring

About New Rebellion

New Rebellion is a marketing intelligence consultancy. We build tools, score Australian businesses on how their marketing actually performs, and publish Debrief every day. This dictionary is part of how we work in the open.

How we think →