Robots.txt Generator

Based on your requirements

Allow all robots to access the entire siteBlock all robots from the entire siteBlock a specific directoryBlock a specific fileAllow only a specific robot (e.g., Googlebot) and block all othersBlock specific URL parametersAllow crawling of a specific directory and block everything elseBlock images from a specific directoryBlock access to CSS and JS files

Start selecting a default configuration

Based on your requirements

Based on your project

Optional configurations

Allow/Disallow AI bots
Block all bots

Sitemap url

Your robots.txt file

Start selecting a default configuration

Based on your requirements

Based on your project

Optional configurations

Allow/Disallow AI bots Block all bots

Sitemap url

Your robots.txt file

Allow/Disallow AI bots
Block all bots