Printable View
i have prepare robots.txt file in which i want to allow all search engines to crawl all pages of my site except three pages. my file looks like following:
User-Agent: *
Disallow:/indexA.html
User-Agent: *
Disallow:/indexA.php
User-Agent: *
Disallow:/index-login.php
Do This file is right or do i have done some mistakes?
First why don't you want search engine to index those pages? You do know that anyone can see your robots.txt and know exactly what pages you want to hideQuote:
Originally Posted by thesheild
i have prepare robots.txt file in which i want to allow all search engines to crawl all pages of my site except three pages. my file looks like following:
User-Agent: *
Disallow:/indexA.html
User-Agent: *
Disallow:/indexA.php
User-Agent: *
Disallow:/index-login.php
Do This file is right or do i have done some mistakes?