User Tag List

1760Views
Results 1 to 2 of 2

Thread: Robots.txt

  1. #1
    Member thesheild's Avatar
    Join Date
    Feb 2018
    Posts
    60
    Coins
    4
    Thumbs Up
    Received: 17
    Given: 6

    Post Robots.txt

    i have prepare robots.txt file in which i want to allow all search engines to crawl all pages of my site except three pages. my file looks like following:

    User-Agent: *
    Disallow:/indexA.html

    User-Agent: *
    Disallow:/indexA.php

    User-Agent: *
    Disallow:/index-login.php

    Do This file is right or do i have done some mistakes?

    0 Not allowed! Not allowed!

  2. #2
    Wind Guardian Angemon's Avatar


    Join Date
    Jul 2014
    Posts
    2,590
    Coins
    813
    Thumbs Up
    Received: 1,238
    Given: 487
    Quote Originally Posted by thesheild View Post
    i have prepare robots.txt file in which i want to allow all search engines to crawl all pages of my site except three pages. my file looks like following:

    User-Agent: *
    Disallow:/indexA.html

    User-Agent: *
    Disallow:/indexA.php

    User-Agent: *
    Disallow:/index-login.php

    Do This file is right or do i have done some mistakes?
    First why don't you want search engine to index those pages? You do know that anyone can see your robots.txt and know exactly what pages you want to hide

    1 Not allowed! Not allowed!



Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •