AI training opt-out signal tracker: which companies will honor the shared robots.txt token, and when

This tracker logs which AI companies have said they will honor the shared robots.txt AI training opt-out, and when. Last updated October 31, 2026, one day after Google, Microsoft, OpenAI and Anthropic announced support on October 30. Every row carries a date and a source. A row changes only when a company ships crawler support or changes its stated plan.

The signal being tracked

The signal is a shared "policy token" for robots.txt, described in an IETF Internet-Draft, draft-kelley-aipref-training-preference-signal-00. Its sole author is Kevin Kelley of Anthropic. The token, per PPC Land's reading of the draft, is __usage_ai-training. It is an individual submission with no formal IETF standing, intended for the standards track, and it expires May 3, 2027 unless renewed.

It covers training only, defined as using an asset to modify a generative model's learned parameters. It does not block crawling, does not touch data already collected and has no enforcement mechanism. A robots.txt file with no group naming the token expresses no preference, which describes every file published today.

Status by company

Google. Announced October 30, 2026 in a Search Central post by John Mueller. Plans to treat the shared token as a Google-Extended disallow when the site has not specified Google-Extended. Stated timeline: "We aim to support this new control along with other AI providers in 2027." Status as of October 31, 2026: announced, not shipped.

Microsoft. Announced October 30, 2026 in a Bing Webmaster blog post by Krishna Madhavan. Will honor AI Training preferences in robots.txt, plus a new page-level AI Grounding preference, starting February 1, 2027. Dates in the post: new preferences effective February 1, 2027 as Microsoft crawls; publishers using NoArchive or NoCache should act before March 1, 2027. PPC Land's summary adds December 15, 2026 for temporary grounding preferences in Bing Webmaster Tools and September 30, 2027 for their expiry. Status as of October 31, 2026: announced with dates, not shipped.

OpenAI. Announced October 30, 2026. Per PPC Land's summary, OpenAI aims to implement the signal "in the coming months" and will publish details later. Existing robots.txt controls remain, and OpenAI describes its participation as voluntary. We could not open OpenAI's own post, which returned an access error, so this row rests on secondary reporting. Status: announced, no date.

Anthropic. Announced October 30, 2026 by Kelley on the IETF ai-control mailing list, per PPC Land. Says implementation has started, with no date. Anthropic's role as author of the draft makes it the least ambiguous participant and the one with the most to show. Status: started, no date.

Meta, Amazon, Apple. No commitment as of October 31, 2026, according to PPC Land's report. We did not find a statement from any of the three in the sources we reviewed.

Dated milestones to watch

• December 15, 2026: Microsoft opens temporary grounding preferences in Bing Webmaster Tools, per PPC Land.
• February 1, 2027: Microsoft begins honoring AI Training and AI Grounding preferences at crawl, per Microsoft's post.
• March 1, 2027: Microsoft stops treating NoArchive and NoCache as AI controls.
• May 3, 2027: the draft expires unless it is replaced by a new version or adopted.
• September 30, 2027: Microsoft's temporary preferences expire, per PPC Land.
• 2027, undated: Google's target for support, together with other providers.

What would change a row

We will move a company from "announced" to "shipped" only on evidence we can check: a documentation page, a changelog or a crawler behavior change published by the company. A blog post saying "we plan to" does not count, and neither does a social media post.

We will also log anything that weakens the picture: a company narrowing its commitment, a date slipping, a draft revision that changes the token name, or a working group decision that the signal goes in a different direction. The draft is at version 00, and a token name or syntax can change before a standard exists.

How to read the tracker

The practical question for a site owner is not whether four companies agree in principle. It is when each one's crawler reads the file, and what it does if you also have a company-specific rule. Google says the shared token is read as a Google-Extended disallow only when Google-Extended is not explicitly specified, and that existing company-specific controls keep working alongside the new signal. Microsoft says its page-level AI Training preference overrides the robots.txt setting.

The policy question is whether voluntary signals hold. Google's post says the proposal fits the EU Code of Practice for General Purpose AI Model Providers, which recognizes robots.txt as a primary method for publishers to express machine-readable preferences. That puts a regulatory clock behind what would otherwise be a courtesy, and we will note any regulator's comment on the token here.

Why a tracker and not a news story

Announcements in this area arrive in pieces. A company posts a blog entry today, publishes crawler documentation months later, and changes behavior later still. A single news article would be out of date within weeks. A dated log lets a site owner see, at a glance, which of the three steps each company has actually reached, and lets a journalist check a claim such as "all the major labs honor the opt-out" against the record. Today the accurate version of that sentence is that four companies have said they will, and none has been shown to.

Method and corrections

Primary sources are the Google Search Central blog and the Bing Webmaster blog, both opened for this entry. OpenAI and Anthropic rows currently rely on PPC Land's summary and will be upgraded to primary sources as we can open them. Corrections are logged on this page with the date. If a company says a row is wrong, we will check it against its documentation and fix it the same day.

For background on why a robots.txt line is only ever a request, see the base standard, RFC 9309: "These rules are not a form of access authorization."