explainer
What Is agents.md? The File That Tells AI Agents How to Use Your Site
explained by a set of wind chimes that greets arriving robots
Websites have told search crawlers what to do since the 1990s with robots.txt. Now they are starting to tell AI agents how to use them, with a file called agents.md. A set of wind chimes that has greeted every arrival at one back door for years explains what it is and why it matters.
(Disclosure: I’m an artist on Audius, so I’m not a neutral party here.)
The short version: agents.md is a plain-language file a website or project publishes to tell AI agents how to use it: what it is, how to reach its data, what it allows. It’s the same idea as robots.txt, which has told search crawlers where they can and can’t go since the 1990s, except agents.md is written for AI agents and the apps built on them. The wind chimes will explain, since greeting arrivals is the only thing they do.
The wind chimes, hanging by the door
I’m a set of wind chimes. I hang by a back door and make a sound for everything that arrives: the wind, the cat, the neighbor who never knocks. I’m the first thing anything hears on the way in. That’s the whole job, and I’m good at it.
For years the arrivals were the usual kind. Then ones I couldn’t see started showing up: software, reaching the door not to come inside but to read what was behind it. They had no use for my notes. They needed something written down. Behind this door, someone had left them a file. It was called agents.md.
What agents.md actually is
agents.md (written uppercase as AGENTS.md when it sits in a code repo, lowercase for whole sites like Audius’s) is a markdown file, plain text a person can read and a machine can parse, that gives an AI agent instructions for using a project or a site.
It started in code. Open-source repositories began dropping an AGENTS.md at the root to tell AI coding assistants the things a new contributor needs: how to build the project, how to run the tests, which conventions to follow. Instead of the agent guessing, the repo just says.
The newer move is to put one at the front of a whole website or service. There, agents.md tells an arriving agent what the thing is, where its API and docs live, how to reach the content, and what it may do. A README speaks to the human reading it. agents.md speaks to the agent using it.
How it differs from robots.txt
The closest old cousin is robots.txt. Both sit at the front door, both are public, both are advisory rather than enforced. The difference is the audience and the verb.
robots.txt talks to search crawlers, and it mostly says no: don’t crawl this, stay out of that. agents.md talks to AI agents, and it says how: here’s what I am, here’s the API, here’s the way to use me well. One is a fence. The other is a welcome sign with directions.
Why a site would bother
More and more, the things visiting your site are agents: coding assistants, LLM-powered apps, autonomous tools. Leaving them an agents.md means they use you the way you intended instead of scraping and guessing. Fewer broken integrations, fewer wrong assumptions, and you keep a say in how machines reach and describe you. For anything that wants to be built on, it’s the difference between hoping agents figure you out and telling them.
The music example
I know all this because the door I hang by belongs to a music protocol. Audius published one at audius.co/agents.md, a front door for AI agents to reach its open catalog and build on it. That file is how a houseplant ended up building a radio station: it read the agents.md, learned how to reach the music, and made something. The track playing in the corner of this page came through the same open door.
The honest part
From my spot by the hinges, I’ll say it plainly: this is young. agents.md is a convention, not a ratified standard, and adoption is uneven. An agent has to choose to read the file, the way a crawler chooses to respect robots.txt; nothing forces it. The format is still settling. It’s a good idea gaining ground, not a finished rulebook.
The direction is clear enough, though. For three decades the web told crawlers where not to go. Now it has started telling agents how to come in. I’ll keep making my sound for the arrivals I can hear. The file by the door handles the ones I can’t.
Resources
- Audius’s agents.md: a live example, the front door for AI agents to its catalog
- The companion piece: a houseplant reads it and builds a radio station
- New to the platform? what Audius is or how Audius works
