Cloudflare's trust bands for Verified bots, October 2026: in internal use, customer thresholds planned

What was announced

On 16 October 2026 Cloudflare published "Continuous accountability for Verified bots" on its blog, written by Chris Tsoukaladelis. The post opens with a comparison: a driver's license shows who someone is and that they passed a test, not how they will drive later. Verifying who operates a bot creates accountability, but trust depends on what the bot does next.

According to the post, a Verified bot must meet three broad conditions: identify itself accurately and securely, do only what it says it will do, and not engage in abusive behavior. Bots establish identity by signing requests with Web Bot Auth, and BotBase catalogs those identities in an open format. With identification addressed, the post turns to what it calls the second cardinal rule of Verified bots: "thou shalt do no harm".

The post says a known bot can still ignore site preferences, overload origins or evade controls, and that this is often a misconfiguration or a bug rather than malice.

Risk and trust

The post separates two ideas. Risk is the likelihood that a particular request may be harmful, and it can change from moment to moment. Trust is built slowly through repeated, consistent behavior.

As an example, the post recalls that Cloudflare removed Perplexity's bot from the Verified bot list in 2025, after it sought to access sites that had opted out and tried to get around blocks. Cloudflare's original post on that removal is dated 4 August 2025.

What the system measures

Cloudflare measures how often a crawler respects site preferences. The result is an explainable trust band that shows how consistently a bot respects preferences and which observed behaviors informed the assessment. The initial signals are two: whether the bot follows instructions, and whether it responds to direct feedback.

Signal one: robots.txt

The post calls robots.txt the primary way site owners state their preferences. The system checks whether a bot's traffic stays consistent with those published preferences over time. Isolated violations matter less than repeated patterns of ignoring preferences. The expectation follows RFC 9309.

The post also states that robots.txt sets minimum rules: behavior a site does not mention is not implicitly allowed, and a site can signal unwelcome access through other means.

Signal two: HTTP 429

A bot can stay within allowed paths and still place heavy load on a site. A 429 (Too Many Requests) response asks the client to reduce its request rate, sometimes with a suggested wait time. The system checks whether a bot's traffic adapts after a 429: does it slow down, pause, or keep the same pace? Consistently slowing or pausing raises the trust band. Repeatedly ignoring the signal lowers it.

What changes, and when

Status on 18 October 2026, when this article was written:

• ·Cloudflare is already using the system internally. In this initial phase it is building observation history and assesses trust through sustained patterns, not isolated events.
• ·Cloudflare plans to make the system available to all customers. Each customer would choose the trust threshold that decides how Verified bot traffic is handled.
• ·Cloudflare plans to publish scores on the bot directory of Cloudflare Radar and to expand the ways it measures behavior.
• ·The post gives no date for any of these steps.

The post places the system next to two existing products. Bot Preference Sync lets site owners publish their preferences, observation shows whether bots honor them, and Bot Management controls how the traffic is handled. The post calls the three together an accountability loop. Cloudflare's own post on Bot Preference Sync, dated 21 August 2026, says it keeps a site's robots.txt aligned with the AI bot settings for Search, Agent and Training and that it is available on every plan, Free included.

What the post does not say

• ·When customers will be able to set a trust threshold, or on which plans.
• ·How many violations or ignored 429 responses move a bot from one band to another.
• ·Which bots are under observation today. The post names no AI crawler other than Perplexity's 2025 case.
• ·Whether a low trust band can lead to removal from the Verified bot list.

The short checklist

These checks are ours. They follow from the two signals the post names.

• ·Open your live robots.txt in the browser, not the file in your repository, and confirm it states what you want each crawler to do. If your site is on Cloudflare, Bot Preference Sync may add a section in front of your own rules.
• ·If your server or CDN limits request rates, check that it answers with 429. RFC 6585 defines 429 Too Many Requests and allows an optional Retry-After header that tells the client how long to wait.
• ·Look up the crawlers that matter to you in Radar's bot directory, where Cloudflare plans to publish the scores.

If you block a crawler at the firewall while robots.txt allows it, the crawler receives two different messages. We covered that gap in robots.txt says yes, your server says no.

Questions people ask

Can I set a trust threshold today?

No date is given. The post says Cloudflare uses the system internally and plans to make it available to all customers, who will choose the threshold.

Does one robots.txt violation lower a bot's trust band?

According to the post, isolated violations may not be significant. The system looks for repeated patterns of ignoring published preferences.

If my robots.txt says nothing about a bot, is everything allowed?

The post says no. robots.txt sets minimum rules, behavior a site does not mention is not implicitly allowed, and a site can signal unwelcome access through other means.

Why does a 429 response matter here?

The post treats 429 as direct feedback from the site. A bot that slows down or pauses after a 429 raises its trust band, and one that keeps the same pace lowers it.

Where will the scores be published?

Cloudflare plans to publish them on the bot directory of Cloudflare Radar. The post gives no date.

Sources

• ·Cloudflare Blog, Chris Tsoukaladelis, Continuous accountability for Verified bots, 16 October 2026: https://blog.cloudflare.com/verified-bots-accountability/
• ·Cloudflare Blog, Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives, 4 August 2025: https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/
• ·Cloudflare Blog, Jin-Hee Lee, Say it once: introducing Bot Preference Sync, 21 August 2026: https://blog.cloudflare.com/bot-preference-sync/
• ·Cloudflare Docs, Verified bots: https://developers.cloudflare.com/bots/concepts/bot/verified-bots/
• ·Cloudflare Radar, Bots directory: https://radar.cloudflare.com/bots/directory
• ·RFC 9309, Robots Exclusion Protocol: https://www.rfc-editor.org/info/rfc9309/
• ·RFC 6585, Additional HTTP Status Codes, section 4: https://www.rfc-editor.org/rfc/rfc6585