Robots.txt Generator
Robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
Share on
Set All to
Crawl delay
Disallow URLs
Search Robots
Google
Google Image
Google Mobile
MSN Search
Yahoo
Yahoo MM
Yahoo Blogs
Ask/Teoma
GigaBlast
DMOZ Checker
Nutch
Alexa/Wayback
Baidu
Naver
MSN PicSearch
Generated Robots.txt