Quillly All crawlers
xAI-Web-Crawler
Operated by xAI · Other AI bots
General xAI web crawler.
User-agent token
xAI-Web-Crawlerrobots.txt
robots.txt compliance not documented
Allow or block xAI-Web-Crawler
By default every well-behaved crawler is allowed. To block xAI-Web-Crawler specifically, add this to your robots.txt:
User-agent: xAI-Web-Crawler Disallow: /
Did xAI-Web-Crawler read your content?
Quillly detects xAI-Web-Crawler and every other crawler server-side when they fetch your published pages, and shows it in your analytics — so you can see exactly which AI picked up which post, and when.
Track your crawlers free