SEOTools.info / Tools / robots.txt Generator
robots.txt Generator
Build a valid robots.txt: choose which bots to allow or block, add disallow paths and your sitemap.
Next step: robots.txt Checker →
About the robots.txt Generator
Builds a valid robots.txt — choose which bots to allow or block, add disallow paths and your sitemap — so crawlers see exactly the rules you intend.
What this tool checks
- Set user-agent, allow and disallow rules
- Add your sitemap URL
- Copy a ready-to-upload robots.txt
Frequently asked questions
Where does robots.txt go?
At the root of your domain, e.g. example.com/robots.txt — crawlers look there first.
Can I block a whole site?
Disallow: / blocks everything for the given user-agent — useful for staging, but never ship it to production by accident.
Put these on autopilot
Generate, monitor and track — the whole toolkit in one panel.