robots.txt Generator
Rule 1
Allow
Disallow
Crawl Delay0s
robots.txt
User-agent: *
Allow: /
Disallow: /admin/
Disallow: /api/
Disallow: /private/
Sitemap: https://example.com/sitemap.xmlHistory
No saved configs yet.
User-agent: *
Allow: /
Disallow: /admin/
Disallow: /api/
Disallow: /private/
Sitemap: https://example.com/sitemap.xmlNo saved configs yet.