[SGVLUG] web server throttling

[Claude Felizardo]

Got an interesting web server question for the list...

How would you plan for a slashdot effect?

Basically one of the projects at work is getting ready to publish some
new data which will be available via a web service.  They expect most
users will run it interactively trying a few queries and download some
data when they find something of interest.  They also expect some
people will write scripts which will try to download as much as they
can.   The project wants to throttle the heavy users so that the
interactive users can use the system as it was designed.

Are there any apache modules that can be used to automatically limit
the number of requests from either a specific IP or block of IPs
without having to explicitly list them in advance.   Need something
that will do this automatically and can be tuned.  Possibly a tunable
parameter that says once you go above X number of requests in a
certain time period then you get a tailored message saying to try
again later.  Don't want to just drop the request as they'll just keep
retrying - we want them to know they are abusing the system.

Apparently there was something like this for Apache 1.x but it hasn't
been updated for 2.x.

Has anyone done anything similar?

claude