User Tag List

3536Views
Results 1 to 2 of 2

Thread: Robots.txt

  1. #1
    Member thesheild's Avatar
    Join Date
    Feb 2018
    Posts
    59
    Coins
    4
    Thumbs Up
    Received: 17
    Given: 6

    Post Robots.txt

    i have prepare robots.txt file in which i want to allow all search engines to crawl all pages of my site except three pages. my file looks like following:

    User-Agent: *
    Disallow:/indexA.html

    User-Agent: *
    Disallow:/indexA.php

    User-Agent: *
    Disallow:/index-login.php

    Do This file is right or do i have done some mistakes?

    0 Not allowed! Not allowed!

  2. #2
    Wind Guardian Angemon's Avatar


    Join Date
    Jul 2014
    Posts
    2,571
    Coins
    813
    Thumbs Up
    Received: 1,186
    Given: 465
    Quote Originally Posted by thesheild View Post
    i have prepare robots.txt file in which i want to allow all search engines to crawl all pages of my site except three pages. my file looks like following:

    User-Agent: *
    Disallow:/indexA.html

    User-Agent: *
    Disallow:/indexA.php

    User-Agent: *
    Disallow:/index-login.php

    Do This file is right or do i have done some mistakes?
    First why don't you want search engine to index those pages? You do know that anyone can see your robots.txt and know exactly what pages you want to hide

    1 Not allowed! Not allowed!



Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •