Meta Tags for Robots Indexing
If you choose not to use a
robots.txt file, you can also allow or disallow robots from indexing the
page or following the links on the page.
Using meta tags to disallow
robots is especially useful if you have hidden directories you do not
want indexed, but that you don't want to alert others about by placing
it in your robots.txt file.
There are four different ways
you can use the meta tag code:
||Index the page
||Do not index the page
||Follow/index all links
on the page
||Do not follow/index any
links on the page
To use the tag, you need to
insert the code into the HEAD content of your html page.
To index the page and follow
To index the page but not follow
To not index the page but follow all links:
To not index the page and not
follow all links:
To use a robots.txt file instead
of metas, you can make your own with our robots.txt
tutorial, or you can use our robots.txt
generator to automatically make your own.