List of Bots

From TNG_Wiki
Jump to: navigation, search

The following list of bots was provided by Jerome Liebowitz on the user list

If a new bot appears that I haven't identified on his mod, I just add it to his modification of log.php (as described in the instructions). As of now, these are the bots I've identified for the separate bot log (code added to log.php):

if (strpos($remhost,"google") !== false || 
		strpos($remhost,"msnbot") !== false || 
		strpos($remhost,"inktomi") !== false || 
		strpos($remhost,"cuill.com") !== false || 
		strpos($remhost,"werelate") !== false || 
		strpos($remhost,"livebot") !== false || 
		strpos($remhost,"Panscient") !== false || 
		strpos($remip,"74.6.") !== false || 	// yahoos inktomi search engine
		strpos($remip,"72.30.") !== false || 	// yahoos inktomi search engine
		strpos($remip,"38.114.104") !== false ||    // Gigibot search engine
            strpos($remip,"38.98.19.") !== false ||         // snap.com bot
		strpos($remip,"38.99.13.") !== false ||     // cuill.com's twiceler search engine
		strpos($remip,"38.99.44.") !== false ||     // cuill.com's twiceler search engine
		strpos($remip,"64.1.215.") !== false ||     // cuill.com's twiceler search engine
		strpos($remip,"208.36.144.") !== false ||   // cuill.com's twiceler search engine
            strpos($remip,"82.99.30.") !== false ||	    // Munax search engine
            strpos($remip,"205.209.188.242") !== false ||
		strpos($remhost,"crawl.yahoo") !== false ||
            strpos($remhost,"search.mud.yahoo.com") !== false ||
            strpos($remhost,"search.dcn.yahoo.net") !== false ||
		strpos($remhost,"picsearch") !== false ||
            strpos($remhost,"ptr.us.xo.net") !== false ||
            strpos($remhost,"become") !== false ||
		strpos($remhost,"entireweb") !== false ||
		strpos($remhost,"myfamilyinc") !== false ||
		strpos($remhost,"accelovation") !== false ||
		strpos($remhost,"accelobot") !== false ||                
		strpos($remhost,"crawler") !== false ||
		strpos($remhost,"crawl") !== false ||
		strpos($remhost,"goo.ne.jp") !== false ||
            strpos($remhost,"phx.gbl") !== false ||
		strpos($remhost,"alexa.com") !== false ||
		strpos($remhost,"biz.arrival.net") !== false ||
		strpos($remhost,"cust.bredbandsbolaget.se") !== false ||
		strpos($remhost,"crawl-4.cuill.com") !== false ||
            strpos($remhost,"amazonaws.com") !== false ||
		strpos($remhost,"craw.blueyonder.co.uk") !== false ||
		strpos($remhost,"spider") !== false || 
		strpos($remhost,"client.mchsi.com") !== false || 
		strpos($remhost,"gigablast") !== false )              
		{
			$thefile = "botgenlog.txt";
		} else {
			$thefile = $logfile;

References on User Agent

Provided by Gustave Dahl on User2 list on 13 Nov 2009