VantacronBot

VantacronBot is the automated crawler that fetches publicly available web pages so Vantacron can run SEO audits requested by our users. This page explains how it behaves and how site owners can control it.

How to identify it

How it behaves

• It only accesses publicly available pages — the same content any browser or search engine can reach.
• It respects robots.txt rules.
• It rate-limits requests per host to avoid placing load on your server.
• It does not attempt to bypass logins, paywalls or access controls.

How to block it

To disallow VantacronBot, add the following to your robots.txt:

User-agent: VantacronBot
Disallow: /

Contact

Questions about our crawler, or a request relating to data accessed during a crawl? Email [email protected].