What Is llms.txt, and Who Actually Reads It

llms.txt is a text file that someone proposed putting at the top level of a website. It lists the pages you most want an AI model to read, so think of it as a map. It is not a citation, and it does not change how you rank.

Cited in this note

28%

Jeremy Howard published the proposal in September 2024, and the page was updated in August 2026.¹ The idea is a short list of pages for a model to read. It should be written so that a person can check it in one sitting. This site already has its own file, and that is a fact about this site, not a traffic number. We publish it because we like having a clean list of what we stand behind.

Google’s guide says that Google does not use llms.txt for its AI features in search.² We keep that next to the proposal, or the proposal turns into a rumor. A file that Google’s AI features do not read cannot be the reason you show up in them. The page still has to be indexed, and it still has to deserve the quote.

To find out whether your file is missing, broken, or pointing at junk, use the missing-file guide is the check. The answer-engine note covers what Google does ask for, which is ordinary search work on a real page. The GEO comparison holds the research on being seen and getting the click.

What a crawl of the files found

Ahrefs published a study in June 2026 that looked at 137,210 domains with website traffic in May 2026.³ About 28% of them had an llms.txt file, which came to about 38,000 valid files. Of those files, 97% got zero requests in May. Of the requests that did arrive, 96% came from bots. Named AI tools made up 19.5% of all requests, and audit or scanner tools made up 12%. We think those shares deserve a pause before you plan a project around the file. In that study most copies were never asked for. When something did ask, it was usually a bot, and only a smaller slice came from named AI tools.

We do not read this as a reason to never publish one. We read it as a reason not to expect the file to be a front door. A list that nobody asks for cannot fix a homepage that says nothing. A scanner asking for the list is not the same as a model citing you in an answer that a customer sees. The study counted requests against files, in one month, on sites that already had traffic. It is not a verdict on the idea, and it only tells us how often the file was fetched.

Publish the file if you want a clean list of your pages, but do not expect Google to read it. Keep your pages indexed and worth citing either way.

That is the order we use when we sit down with a client. Pages come first, then getting them indexed, then a sentence that a stranger can use. After that, if you still want the list, make a short file that points at those pages. Leave out drafts, tag dumps, and anything you would be ashamed to see quoted. The file is a courtesy and a way to keep control, but it is not the real work.

A worked example

Take an independent bookshop. This is an example and not a study, so it has no score. A manager runs the floor and the website on the same tired afternoons. People ask, online and at the desk, for a book for a grieving friend. They ask for something a nine-year-old will really finish, and they ask whether tonight’s reading is still on. The site has a homepage and an events page that is sometimes out of date. It also has a staff-picks page, which is the best thing they have written all year.

Someone on a forum told the manager that models will not understand the shop without llms.txt. So he made a long file. It linked every tag, every old event, and a draft page about a book club that never started. It looked thorough, but it was a bad map. A model that fetched it would have been sent to old nights and a club that does not exist. A model that never fetched it would have changed nothing, and that is the common case in the Ahrefs count. Either way, the manager would have spent his evening on the wrong thing.

If they publish a file, we would want it short. It would hold the staff-picks page. It would also hold the page that explains how to order a book the shop does not have on the shelf. It would hold the events page too, but only if the manager updates it whenever a reading is canceled. It would leave out the tag archives and the draft. The proposal, written in September 2024 and updated in August 2026, is a list of the pages you want read. The key word is “want.” It does not mean everything the website software can produce.

Clear jobs keep the file honest. The owner decides which pages stand for the shop. If she would not hand a page to a regular, it stays off the list. The manager keeps the events page true, because a file that points at a lie is worse than no file. The real audience is not whoever fetches the file, or fails to fetch it. It is the person who needs a book this afternoon. So the staff-picks page has to work on its own, even if no model ever asks for the map. That is the lesson in the request counts. In the study, 97% of the valid files got zero requests in May 2026. A shop that only improved the file and left the picks page mushy would have improved the thing almost nobody asked for.

Google sits to the side of this scene on purpose. The guide says that Google does not use llms.txt for its AI features in search. So the owner could publish a lovely file and still be missing from those features. That happens if the picks page is thin, blocked, or hard to quote. The answer-engine work is the page, and the file is optional housekeeping. We would tell the manager that plainly. Then he will not buy a plugin that rebuilds the list every night from every web address it can see.

The mix of requests matters too, when someone waves a log at you. In that June 2026 study, 96% of the requests that did arrive came from bots. Named AI tools made up 19.5% of the requests, and audit or scanner tools made up 12%. A hit in a log is not a customer, since it is often a machine checking whether the file exists. Count a scanner hit as a scanner hit, and do not tell the owner it is demand.

How we’d run it

We would first decide whether the list is worth keeping. Then we would write it by hand the first time.

First, pick the pages you hope a stranger finds, with the owner in the room. For the bookshop that means the staff picks and how to order a title the shop does not stock. It could also mean tonight’s reading, but only if that page stays current. If you cannot name them without opening the website editor and scrolling, you are not ready to publish a map.

Second, make those pages good enough to quote with no file at all. They should be indexed, and they should open with clear lines. They should also have a sentence the manager would say at the desk. Google does not use the file for its AI features. So this step is the one that can affect that part of search. The answer-engine note says more about what good enough means. Please do not skip it just because a file feels more solid.

Third, write a short llms.txt that points only at those pages. Give each one a plain line of description that a person can check. Link the live address, and if the events page is out of date, leave it off until the manager fixes it. A short true list beats a long machine-made one. This site’s own file was a choice of that kind, and it is not proof that files bring visits.

Fourth, put the file at the top level of the site, and then let go of the dream. Ahrefs looked at 137,210 domains with traffic in May 2026. It found that 28% had a file, about 38,000 valid ones, and 97% of those saw zero requests that month. If yours is in the quiet majority, the shop is not broken. The pages are still what matters.

Fifth, if you do watch your logs, tell machines apart from people. In the study, 96% of requests came from bots. Named AI tools made up 19.5%, and audit and scanner tools made up 12%. A spike from a scanner is not a crowd of readers, so do not rewrite the shop around it.

Sixth, look at the list again when the shop changes, and not on a timer. A new picks page, a book club that ended, or a reading series that wrapped up all count. The owner takes out the dead link that same week. A file that points at last year’s calendar sends people the wrong way with total confidence. The August 2026 update of the proposal did not change that duty. The list should say what is true, or it should come down.

Common mistakes

The first mistake is expecting Google to fetch it. The guide says Google does not use llms.txt for its AI features in search. So publishing one and waiting for AI answers is a plan the guide already turned down. Fix the page that Google can index instead.

The second mistake is a file made by a machine that dumps everything. Every tag, every attachment, and every draft ends up on it, but the proposal is a picked list. A dump hides the staff-picks page in noise, which is the opposite of pointing a model at your best work. It also goes stale, because nobody will look after a thousand automatic lines. We would rather the manager keep up a list that he can read aloud.

The third mistake is treating the Ahrefs shares as a personal failure or a trophy. Of those 137,210 domains, 28% had a file, which came to about 38,000 valid files, and 97% of them got zero requests in May 2026. You are not behind because you lack one. You are not ahead because you have one that nobody asked for. The study describes a crawl of files and says nothing about your customers.

The fourth mistake is reading bot traffic as people taking it up. Of the requests that arrived, 96% came from bots. Audit or scanner tools made up 12%, and named AI tools made up 19.5%. None of that says a reader found the shop, so call them machine requests or leave them out of the report.

The fifth mistake is letting the file disagree with the shop. Say the events page says Thursday, but the blurb in the file says Wednesday, copied from an old night. Whichever one gets quoted, you look careless. Keep the live page as the one place the facts live, and let the file follow it.

The sixth mistake is doing the file instead of the page because the file is smaller. A map to a vague homepage still leads to a vague homepage. The pages must be worth quoting whether or not anyone ever asks for the file. That is the plain lesson of a study where almost all valid files sat untouched for the month they measured.

What we’d do

We would publish a short file only after the pages are true. We would not tell the owner it will bring her readers. Our view is that the proposal is a fair courtesy and a poor priority. September 2024 is when the idea was written down, and August 2026 is when the page was updated. In June 2026 a large study of 137,210 domains found that 97% of valid files got zero requests. It also found that 96% of the requests that did show up came from bots. A courtesy is allowed to be quiet, but a plan is not allowed to ignore a quiet result.

Google has said it does not use the file. So we would spend the manager’s evening on the staff-picks page, the ordering page, and an events page that survives a canceled reading. Then we would add a list of those addresses if the owner likes to see what the shop stands behind in one place. We would keep our own file for that same reason, and we would never wave it around as proof of traffic. A file is a pointer, and the page is what a person came for. If you only have time for one, choose the page. The pointer can wait until the page can stand on its own.