Quillly All crawlers
SemrushBot
Operated by Semrush · SEO tools
Backlink, keyword and rank data for the Semrush toolset (also covers the SemrushBot-* variants).
User-agent token
SemrushBotrobots.txt
Documented to honour robots.txt
Allow or block SemrushBot
By default every well-behaved crawler is allowed. To block SemrushBot specifically, add this to your robots.txt:
User-agent: SemrushBot Disallow: /
Official Semrush documentation
Did SemrushBot read your content?
Quillly detects SemrushBot and every other crawler server-side when they fetch your published pages, and shows it in your analytics — so you can see exactly which AI picked up which post, and when.
Track your crawlers free