Need some IP help/explaination

Mr Furious · 10-01-2010, 21:46

Need a computer savvy person to interpret the below taken from website log file. Two different sets that I am very interested in. Thanks for any help! Mr F.

(First set)

95.108.150.235 - - [01/Oct/2010:11:49:18 -0500] "GET /robots.txt HTTP/1.1" 404 1207 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"
95.108.150.235 - - [01/Oct/2010:11:49:18 -0500] "GET /robots.txt HTTP/1.1" 404 1207 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"
95.108.150.235 - - [01/Oct/2010:11:49:19 -0500] "GET / HTTP/1.1" 200 19658 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"
95.108.150.235 - - [01/Oct/2010:11:49:19 -0500] "GET / HTTP/1.1" 200 19658 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"
95.108.150.235 - - [01/Oct/2010:11:49:20 -0500] "GET / HTTP/1.1" 200 19658 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"
95.108.150.235 - - [01/Oct/2010:11:49:20 -0500] "GET / HTTP/1.1" 200 19658 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)"

(Second set)

88.190.11.232 - - [01/Oct/2010:18:07:47 -0500] "GET /acotacm.html HTTP/1.0" 200 13724 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.9) Gecko/20100824 Firefox/3.6.9 GTB7.1"

18Z · 10-02-2010, 10:51

Not at all sure what your looking for but here is some info

Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots) — [COLOR="Lime"]robot used to determine site mirrors;[/COLnfoOR]

More info....http://help.yandex.com/search/?id=1112612

BigJimCalhoun · 10-02-2010, 12:53

It depends on what the fields are for your web server logging and the logging format. Various options can be set to capture various things.

Item 1
The first part is the IP address of the client who came to your site
It look for your robots.txt which is what a search bot often looks for to determine if it is allowed to crawl your site. It was returned a HTTP 404 error meaning the page was not found. The last part appears to be the "user agent" which seems to be a crawler. Essentially, something appears to be indexing your site. If you don't want indexing a robots.txt can be added with specific options telling robots not to index, crawl.

Second set
client came in from that IP address, got an HTTP 200 response, which is a successfully returned page. I "think" it took 13724 milliseconds to return the response, or it could be bytes in size of the page. Again, it depends on your log format and what you are capturing. The last part is the user agent, Windows NT 6.1 == Windows 7, French version of Firefox.

see http://www.useragentstring.com/Firef...9_id_16360.php

As you stated you are very interested in knowing more, perhaps you have the ability to change your log format to get more information. The link below is for a Microsoft IIS web server, but it details the fields of the W3C format which is not MS specific. The link shows what can be logged. Depending on your webserver, the steps to include this vary, but at least on Microsoft IIS, it is a trivial set of steps.

http://www.microsoft.com/technet/pro....mspx?mfr=true

Mr Furious · 10-02-2010, 13:15

Thank you both, and it does explain things a bit for me. I appreciate that. Since these IP's have shown up on a few of the logs we are curious as to "who" in Moscow and "who" in Paris is actually hitting the site. Yes, I can configure what we capture and also have other analytics embedded in the site. Haven't been able to go too far down the rabbit hole, and that's why I'm reaching out.

BigJimCalhoun · 10-02-2010, 14:40

There are also proxy services one can subscribe to where your IP goes through several sites around the world prior to its destination, so when your actual IP is masked but appears to the web server as originating in AU or Japan or other location.

Mr Furious · 10-02-2010, 23:19

18Z, thanks for the link. I didn’t realize Yandex originated from RU. It just appears to be indexing the site.

BigJim thanks for the description. FYI – we use Plesk and have analytics built in. I can trace although not reliably with CGI. Thanks!

Irishsquid · 12-27-2010, 13:07

IIRC, Yandex.ru is considered a "known-bad actor," and a malware source site. I haven't looked deeply into what they are or do, though I will later, when I'm not on a .mil network. I'll see what I can find out.

As an aside, it might not be a bad idea to get a decent IDS in place, as server logs only tell so much. Then again, I'm a network analyst, so I may be a bit biased. I like to be able to break things down to the packet level and see what's going on.

As a general rule, I avoid, like the plague, anything ".ru," as the Russians have STRONGLY embraced cybercrime as THE new way to make money.

perdurabo · 12-27-2010, 13:23

In this specific example, it's just harmless indexing bots.

You should also see a number of entries in your logs with legitimate attack attempts for various web software. This is unfortunately a normal part of being a web server, these days.

The key is to keep your software updated and analyze your logs (which it looks like you're already doing) for entries that relate to software you have installed.

Vulnerability scans for Wordpress (blogging software), IIS (Microsoft's web server, which is reasonably secure by default these days) and Cacti (network monitoring) are two web apps I see hundreds of thousands of hits from every day.

Irishsquid · 03-25-2011, 14:27

Quote:

Originally Posted by perdurabo

IIS (Microsoft's web server, which is reasonably secure by default these days)

Only if you keep it patched. Right out of the box, it's still vulnerable to a LOT of stuff. This brings us to our next lesson:

Don't take too long to patch stuff. Here's how a zero-day exploit works (for the non-techies):

1) MS releases a patch.
2) hacker downloads the patch
3) hacker reverse engineers the patch (takes it apart to see how it works, and what it does)
4) If he can find what the patch fixes, he knows the vulnerability.
5) Write an exploit to attack that vulnerability

If he can do this within a few days after patch release, he can own millions of boxes, since most users are lazy with patching. Corporate networks take a long time to patch for a different reason, which brings us to another lesson:

Test patches before you install them on a production server. I use virtual machines for patch testing. Install the patch on a machine that doesn't affect business. Make sure it works. DON'T TAKE TOO LONG TESTING PATCHES. While you're testing it, hackers are writing exploits, and if you take too long, they win.

(My response is a bit of a ramble, and for that, I apologize. I'm tired, and running on nothing but coffee. I'm trying to get information on the screen before I lose my train of thought. If anyone has questions, feel free to ask.)

Kit Carson · 03-25-2011, 20:34

I'm just a DAT and knifemaker but have learned quite a bit about tracking IPs and spammers on our knife and gun forum.

Here are some of the sites that can help find out about IPs and email addresses.

http://www.projecthoneypot.org/home.php

http://www.stopforumspam.com/spamdomainsandips

http://www.botscout.com/

Hope it's ok to post these sites. If I screwed up posting them, please delete.

badshot · 03-26-2011, 03:13

Quote:

Originally Posted by Mr Furious

Need a computer savvy person to interpret the below taken from website log file. Two different sets that I am very interested in. Thanks for any help! Mr F.
"

MF,

You can check IP's yourself at http://whois.arin.net/rest/net/NET-63-216-0-0-1/pft
Paste or type in the box in upper right corner, replace the last number with a zero. Example: 95.108.150.235 would be 95.108.150.0 (also known as class C
address block) and that will show you whom it is.

As others have pointed out bots are generally harmless. Google uses one for its search engine. You can Google robots.txt for how to limit (generally) their presence on your site, its easy.

Something off-topic but relevant that ps admins may know about or be interested in:

Instead of entering a long list of IP's in htaccess file (or router/firewall) to block spammers whom create accounts and post porn links, etc. If you have a
good router/firewall set the TCP_OTHER (generally highly fragmented packet under Advanced TCP settings, or TCP OTHER) to drop or reject the packet. I've noticed several appear to use this technique and it stops them dead.

Hope that helped some MF