Using tngrobots.php

The following was provided by Darrin on the tnguser2 list on 2 Feb 2015

TNG provides a tngrobots.php file that attempts to group the public pages into three groups:

1. Don't index or follow
2. Index but don't follow
3. Index and follow (default)

If you edit that file, you should see how that works, and it should be fairly intuitive how to move pages around. In the end, however, different crawlers will make their own decision about whether to follow these guidelines or not.

By including the file at the beginning of genlib.php, the code in tngrobots.php gets executed at that point. Every public page calls genlib.php, so every page runs through that code, which generates the following for no index and no follow:

<meta name="robots" content="noindex,nofollow" />

For example, if you do not want bots to follow the links from the Family Chart Mod and Maternal And Paternal Lines mod you would add

        case "family":
        case "xerxxIndividMaternalLine":
        case "xerxxIndividPaternalLine":

to the section that generates noindex, nofollow using the $flags['norobots'] = $NOINOF;

Related Links

The following provide additional security measures:

Controlling Site Access

• Using tngrobots.php
• Robots txt
• How to setup a robots.txt file
• Bot Trap
• Htaccess
• Htaccess Deny
• Htaccess Rewrite

Protecting Resources

• Permissions Explained
• Database User
• Move your configuration files
• Move your backup files
• Move your gedcom files

• Overlaid Subroot how to recover from subroot.php overlay
• Prevent Directory Listing
• Protecting access log

• Turn off global register and other PHP overrides

Checking your site for Malware

• Sucuri SiteCheck