Verified Bots Policy
In order to be listed by Cloudflare as a Verified Bot, your bot must conform to the following requirements.
Bot Policy
Minimum Traffic
A bot or proxy must have a minimum amount of traffic for Cloudflare to be able find it in the sampled data. The minimum traffic should have more than 1000 requests per day across multiple domains.
Minimum Zones
Service must be made for a widespread use of zones.
Bot Identification
The user-agent with the following requirements:
- Have at least 5 characters.
- Must not contain special characters.
- Must not include the same user-agent of another verified service.
Domain Owner Consent
Domains should only be crawled with the explicit or implicit consent of the zone’s owner or terms of use. Search engines crawlers must read the robots.txt
to exclude paths to crawl from the owner.
Service Purpose
The purpose of the service should be benign or helpful to both the owner of a zone and the users of the service. The service cannot perform any of the following:
- Bot tooling
- Scrapers
- Scalpers
- Credential-stuffing
- Directory-traversal scanning
- Excessive data scrapping
- DDoS botnets
Crawling Etiquette
The crawling etiquette should check robots.txt
if crawling the whole website. The service should have a rate limit of 1 request per second and it should not attempt to crawl sensitive paths.
Public Documentation
The bot must have publicly documented expected behavior or user-agent format.