Robots.txt Generator
Build a robots.txt file with per-bot rules. Block AI crawlers, set crawl delays, define sitemaps.
Quick add:
Rule group 1
robots.txt rules explained
The robots.txt file lives at the root of your domain and tells well-behaved web crawlers which pages they may or may not index. User-agent: * applies to all bots; specific agents like Googlebot or GPTBotoverride the wildcard. Disallow: /private/ blocks that path; Allow: / explicitly permits it. The Crawl-delay directive asks bots to wait N seconds between requests (Googlebot ignores it). The Sitemap: line helps Google discover all your pages. For generating a sitemap, see the XML Sitemap Generator.
Private & free — this tool runs entirely in your browser.
Recommended: Bluehost — Reliable WordPress & web hosting — exclusive pricing from $3.79/mo.affiliate
Related Web & SEO tools
Meta Tag Generator
Build SEO title, description, and viewport meta tags.
Open Graph Generator
Generate Open Graph and Twitter Card meta tags.
UTM Campaign URL Builder
Append UTM parameters to build trackable campaign URLs.
Hreflang Tag Generator
Generate hreflang link tags for multilingual SEO.
Sitemap XML Generator
Build an XML sitemap from a list of URLs.
SERP Snippet Preview
Preview how your page title and meta description appear in Google search.
Meta Tag Analyzer
Paste a page's HTML head to audit its title, meta, and social tags.
Keyword Density Checker
Count word frequency and keyword density in your content.