thesheild
08-02-2018, 04:40 PM
i have prepare robots.txt file in which i want to allow all search engines to crawl all pages of my site except three pages. my file looks like following:
User-Agent: *
Disallow:/indexA.html
User-Agent: *
Disallow:/indexA.php
User-Agent: *
Disallow:/index-login.php
Do This file is right or do i have done some mistakes?
User-Agent: *
Disallow:/indexA.html
User-Agent: *
Disallow:/indexA.php
User-Agent: *
Disallow:/index-login.php
Do This file is right or do i have done some mistakes?