Quillly All crawlers
AhrefsSiteAudit
Operated by Ahrefs · SEO tools
Technical crawl of a single site, run on demand when its owner starts an Ahrefs Site Audit.
User-agent token
AhrefsSiteAuditrobots.txt
Documented to honour robots.txt
Allow or block AhrefsSiteAudit
By default every well-behaved crawler is allowed. To block AhrefsSiteAudit specifically, add this to your robots.txt:
User-agent: AhrefsSiteAudit Disallow: /
Did AhrefsSiteAudit read your content?
Quillly detects AhrefsSiteAudit and every other crawler server-side when they fetch your published pages, and shows it in your analytics — so you can see exactly which AI picked up which post, and when.
Track your crawlers free