robots.txt tells a crawler what it is allowed to fetch. It says nothing about what actually matters once it gets there. An llms.txt file is a newer, much smaller file that fixes that gap: a plain-text map of your most important pages, written for AI systems instead of search engines, sitting at yoursite.com/llms.txt with no login and no JavaScript required to read it.
A different job than robots.txt
robots.txt is a permissions file. It tells a crawler which folders it can enter and which it cannot, and that is the whole job. It has nothing to say about which of the pages it is allowed to read are actually worth reading.
An AI system trying to answer a question about your business has to work that out on its own, by crawling everything and guessing which pages are current, which are old marketing copy, and which are the real answer. llms.txt removes the guesswork. It is a short, curated list you write yourself: here are the pages that matter, here is one line on what each one covers.
What a real one looks like
The format is deliberately plain. One heading with your organisation name, one summary line describing what you do, then a handful of sections, each a short list of links with a one-line description. No schema to learn, no CMS plugin, no build step. It is a static text file you can write by hand in ten minutes.
Kelriva's own llms.txt opens with a one-paragraph summary of the business, then a "Who We Are" section, a plain list of the three services, and a "Key Definitions" section that answers the exact questions a prospect is likely to type into ChatGPT or Perplexity: what is Intelligent Document Processing, what is an AI Readiness Assessment, what does it cost. That last part is the actual point of the file. It is not a sitemap. It is a set of direct, quotable answers, written once, in the place an AI system already knows to look.
Why this exists now
AI crawlers have a harder problem than search crawlers did. A search engine can index a page, rank it, and let the user judge relevance by reading the result themselves. An AI system has to synthesise an answer on the user's behalf, right now, from whatever it can find. Feeding it your full site and hoping it picks the right paragraph is a weaker strategy than handing it a short, honest map and letting it go straight to the source.
The standard is still early. Not every AI crawler reads llms.txt yet, and adoption is uneven across platforms. That is exactly why it is worth doing now rather than later: the businesses that have one today are a small group, and being in that group before it is standard practice is a real advantage, not a wasted afternoon.
Does your site need one right now
Honestly: it costs almost nothing and it cannot hurt. Writing one takes an afternoon, not a sprint. Having one signals the same thing a clean, well-structured robots.txt used to signal in 2010, that someone technical is actually paying attention to how machines read the site. The businesses without one are not skipping it on purpose. Most have simply never heard of it.
How to check if yours is missing, or wrong
Type yoursite.com/llms.txt into a browser. If it 404s, you do not have one. If it loads, read it the way an AI system would: does the summary line actually describe what you do, are the linked pages still live, and would a stranger understand your three most important services from this file alone with no other context. A stale llms.txt, one written months ago and never updated, is a smaller problem than having none, but it is still worth a five-minute check whenever the site's core offering changes.
Where this fits
An llms.txt file is one piece of a larger question: what does an AI system actually see, say, and get right about your business when someone asks it. That is the same question behind how to choose an enterprise AI consultancy in the first place, and it is the one our AI Visibility Score is built to answer directly: exactly what ChatGPT, Gemini, and Perplexity currently say about your business, verbatim, dated, and checked against what is actually true, before you touch any file.