Robots txt

From TNG_Wiki
Revision as of 22:46, 28 January 2015 by KenRoy (talk | contribs) (added Security Related Links)

Jump to: navigation, search

The Robots txt.org site provides explanation on using the The Robots Exclusion Protocol.

Note that malware or email harvesting bots will ignore the directives of the robots.txt file.

Link provided by BruceM on User2 list 6/30/2009 10:34 PM

Related links

The following provide additional security measures:

Controlling Site Access

Protecting Resources

Checking your site for Malware