Any complaints or comments can be e-mailed to:
The TechFerret spider looks for the following to determine its actions with your web pages.
INDEX and FOLLOW are assumed to be on by default and are not required in the META NAME="robots" tag or X-Robots-Tag header listed below.
Place this <META NAME="robots" CONTENT="noarchive"> tag in the head section of your pages to have TechFerret not display a link to a cache of the page in TechFerret search results. Alternatively, you may put noarchive in the X-Robots-Tag header to have TechFerret not display a link to a cache of the page in TechFerret search results. For example X-Robots-Tag: noarchive
Place this <META NAME="robots" CONTENT="noindex"> tag in the head section of your pages to tell TechFerret not to index the page. Alternatively, you may put noindex in the X-Robots-Tag header to tell TechFerret not to index the page. For example X-Robots-Tag: noindex. This will stop TechFerret from displaying the page in search results.
Place this <META NAME="robots" CONTENT="nosnippet"> tag in the head section of your pages to tell TechFerret not to display a summary of the page in search results. Alternatively, you may put nosnippet in the X-Robots-Tag header to tell TechFerret not to dispaly a summary of the page in search results. For example X-Robots-Tag: nosnippet.
To prevent the TechFerret spider from following a link, add the rel=nofollow attribute to the HTML tag. To prevent the TechFerret spider from following all the links on a page on your site, add the <META NAME="robots" CONTENT="nofollow"> tag in the head section of your page. Alternatively, you may put nofollow in the X-Robots-Tag header to prevent the TechFerret spider from following the links on a page. For example X-Robots-Tag: nofollow.
Add the Crawl-delay instruction to your robots.txt file to add a delay between page requests from your site by TechFerret's spider. For example, Crawl-delay: 10 would add a 10 second delay between page requests by TechFerret.
