Robots.txt file

Search Engines  |  META Tags  |  Search Results  |  Optimization  |  Robots.txt  |  Meta Refresh

Google and the majority of the search engines have stopped using Meta tags as indexing objects.
<meta name="robots" content="all">
<meta name="revisit-after" content="15 days">

That process has been replaced by a text file inserted in the root directory of the web site called "robots.txt".

User-agent: *
Disallow: /cgi-bin/
Disallow: /brad/
Disallow: /familytrip.html

The first line tells the search engine bot that it is welcome to index the site.
You then list any folder or file that you do not want indexed.

Example Results:

Top of Page