[SGVLUG] Fun of running a Website.

[Michael Proctor-Smith]

On 8/16/06, Emerson, Tom <Tom.Emerson at wbconsultant.com> wrote:
> > -----Original Message----- Of Michael Proctor-Smith
> >
> > Yes The sgvlug site is down. I took it down because of an
> > attack. Well at least we got crawled by msn/yahoo/and someone
> > at texas A&M in the last three days(I was reading the logs).
The reverse dns on the A&M one was something.crawler.cs. texas A&M's
domain. So my guess is some kind CS project.

> Ummm... Crawling isn't an "attack" (that I know of) -- not honoring
> robots.txt, however, might be an actionable offense (umm, you HAVE setup
> an appropriate robots.txt file, haven't you?)

To clearify I was not saying that crawling was the attack. It was the
good news. I just noticed when I was looking for the source of
hundreds of perl proccesses calling themselves "https" and using 100%
of the cpus.  There was a load of 45 when I logged in around noon.

Maybe it is just a bug in mambo that is causeing the load and defunct
processes but when I searched for part of the string that was part of
the wierd requests that I found google reported it looked like a
spam/virus attack attempt.

> Msn, yahoo, google, and others will generally be grabbing/caching pages
> for their search engines -- this is "a good thing" (right, Chris?)
> unless there is a loop (and even if there was) these spiders shouldn't
> be consuming huge amounts of bandwidth as they should hit each page and
> move on.  I don't think they bother with images (well, save for google
> for their google images search page -- do the others do the same now?)
>
> That leaves the Texas A&M site -- could be someone did a wget?  Is there
> any indication of attempts to exploit mambo/joomla?
>
> For that matter, didn't Matthew G. install a java-based IRC client?
> Could that have had "a side effect"?  (maybe that's why I couldn't
> connect?)  Are we sure it's been configured correctly?

I checked that first as I noticed it first yesterday after I gave Matt
a login and checked that first. Does not seems to be the problem.