Go Back   Professional Soldiers ® > Technical FAQ Forum > KnuckleDragger Questions

Reply
 
Thread Tools Display Modes
Old 10-01-2010, 21:46   #1
Mr Furious
Quiet Professional
 
Mr Furious's Avatar
 
Join Date: Dec 2009
Location: Center of the Universe, NC
Posts: 652
Need some IP help/explaination

Need a computer savvy person to interpret the below taken from website log file. Two different sets that I am very interested in. Thanks for any help! Mr F.

(First set)

95.108.150.235 - - [01/Oct/2010:11:49:18 -0500] "GET /robots.txt HTTP/1.1" 404 1207 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"
95.108.150.235 - - [01/Oct/2010:11:49:18 -0500] "GET /robots.txt HTTP/1.1" 404 1207 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"
95.108.150.235 - - [01/Oct/2010:11:49:19 -0500] "GET / HTTP/1.1" 200 19658 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"
95.108.150.235 - - [01/Oct/2010:11:49:19 -0500] "GET / HTTP/1.1" 200 19658 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"
95.108.150.235 - - [01/Oct/2010:11:49:20 -0500] "GET / HTTP/1.1" 200 19658 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"
95.108.150.235 - - [01/Oct/2010:11:49:20 -0500] "GET / HTTP/1.1" 200 19658 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"



(Second set)

88.190.11.232 - - [01/Oct/2010:18:07:47 -0500] "GET /acotacm.html HTTP/1.0" 200 13724 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.9) Gecko/20100824 Firefox/3.6.9 GTB7.1"
Mr Furious is offline   Reply With Quote
Old 10-02-2010, 10:51   #2
18Z
Quiet Professional
 
18Z's Avatar
 
Join Date: Jun 2010
Location: Clarksville, TN
Posts: 16
Not at all sure what your looking for but here is some info

Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots) — [COLOR="Lime"]robot used to determine site mirrors;[/COLnfoOR]

More info....http://help.yandex.com/search/?id=1112612
18Z is offline   Reply With Quote
Old 10-02-2010, 12:53   #3
BigJimCalhoun
Guerrilla
 
Join Date: Mar 2009
Location: DFW Area
Posts: 401
It depends on what the fields are for your web server logging and the logging format. Various options can be set to capture various things.

Item 1
The first part is the IP address of the client who came to your site
It look for your robots.txt which is what a search bot often looks for to determine if it is allowed to crawl your site. It was returned a HTTP 404 error meaning the page was not found. The last part appears to be the "user agent" which seems to be a crawler. Essentially, something appears to be indexing your site. If you don't want indexing a robots.txt can be added with specific options telling robots not to index, crawl.


Second set
client came in from that IP address, got an HTTP 200 response, which is a successfully returned page. I "think" it took 13724 milliseconds to return the response, or it could be bytes in size of the page. Again, it depends on your log format and what you are capturing. The last part is the user agent, Windows NT 6.1 == Windows 7, French version of Firefox.

see http://www.useragentstring.com/Firef...9_id_16360.php


As you stated you are very interested in knowing more, perhaps you have the ability to change your log format to get more information. The link below is for a Microsoft IIS web server, but it details the fields of the W3C format which is not MS specific. The link shows what can be logged. Depending on your webserver, the steps to include this vary, but at least on Microsoft IIS, it is a trivial set of steps.

http://www.microsoft.com/technet/pro....mspx?mfr=true
BigJimCalhoun is offline   Reply With Quote
Old 10-02-2010, 13:15   #4
Mr Furious
Quiet Professional
 
Mr Furious's Avatar
 
Join Date: Dec 2009
Location: Center of the Universe, NC
Posts: 652
who

Thank you both, and it does explain things a bit for me. I appreciate that. Since these IP's have shown up on a few of the logs we are curious as to "who" in Moscow and "who" in Paris is actually hitting the site. Yes, I can configure what we capture and also have other analytics embedded in the site. Haven't been able to go too far down the rabbit hole, and that's why I'm reaching out.
Mr Furious is offline   Reply With Quote
Old 10-02-2010, 14:40   #5
BigJimCalhoun
Guerrilla
 
Join Date: Mar 2009
Location: DFW Area
Posts: 401
There are also proxy services one can subscribe to where your IP goes through several sites around the world prior to its destination, so when your actual IP is masked but appears to the web server as originating in AU or Japan or other location.
BigJimCalhoun is offline   Reply With Quote
Old 10-02-2010, 23:19   #6
Mr Furious
Quiet Professional
 
Mr Furious's Avatar
 
Join Date: Dec 2009
Location: Center of the Universe, NC
Posts: 652
18Z, thanks for the link. I didn’t realize Yandex originated from RU. It just appears to be indexing the site.

BigJim thanks for the description. FYI – we use Plesk and have analytics built in. I can trace although not reliably with CGI. Thanks!
Mr Furious is offline   Reply With Quote
Old 03-26-2011, 03:13   #7
badshot
Guerrilla Chief
 
badshot's Avatar
 
Join Date: Apr 2010
Location: Southern Arizona
Posts: 590
Quote:
Originally Posted by Mr Furious View Post
Need a computer savvy person to interpret the below taken from website log file. Two different sets that I am very interested in. Thanks for any help! Mr F.
"
MF,

You can check IP's yourself at http://whois.arin.net/rest/net/NET-63-216-0-0-1/pft
Paste or type in the box in upper right corner, replace the last number with a zero. Example: 95.108.150.235 would be 95.108.150.0 (also known as class C
address block) and that will show you whom it is.

As others have pointed out bots are generally harmless. Google uses one for its search engine. You can Google robots.txt for how to limit (generally) their presence on your site, its easy.

Something off-topic but relevant that ps admins may know about or be interested in:

Instead of entering a long list of IP's in htaccess file (or router/firewall) to block spammers whom create accounts and post porn links, etc. If you have a
good router/firewall set the TCP_OTHER (generally highly fragmented packet under Advanced TCP settings, or TCP OTHER) to drop or reject the packet. I've noticed several appear to use this technique and it stops them dead.

Hope that helped some MF
__________________
Δεν είμαι άξιος του σταυρού του Ιησού οπή, Andreas
Denial and inactivity prepare people well for roles of victim and corpse
badshot is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump



All times are GMT -6. The time now is 16:03.



Copyright 2004-2022 by Professional Soldiers ®
Site Designed, Maintained, & Hosted by Hilliker Technologies