Rate Limiter Application

Last modified by Thomas Mortagne on 2017/09/18 00:04

cogProvides rate limiting services to prevent abusive site grabbing
TypeXAR
CategoryApplication
Developed by

Denis Gervalle

Active Installs3
Rating
Rate!
0 Votes
LicenseGNU Lesser General Public License 2.1
Compatibility
Should be only installed on the main wiki
Installable with the Extension Manager

Description

The rate limiting service provides a way to control how fast users can consume your website content and prevent abusive site grabbing. You can define one or more allow consumption rates for different periods of time. By default, the service install the following limitations:

  • 250 requests  / 15 mins (with an overflow of 1000 requests)
  • 500 requests  / 8 hours (with an overflow of 1500 requests)
  • 5000 requests / 30 days (with an overflow of 5000 requests)

There are maximum number of requests allowed during the given period of time. Requests are not strictly equivalent to pages. Loading a page requires at least one request, but could generate more requests to load images in the page, display attachments, etc... The overflow allows to limit the penalty of exceeding the limit. For example, in the above limitation, user abusing the 15 mins limits will not be delayed longer than an hour (4 times the 15 mins limit, 250 x 4 = 1000) if they persist accessing the site after the limit is reached. User exceeding limits will receive a warning screen similar to:

exhausted.png

If they persist, the waiting time will increase accordingly up to reaching the overflow.

This extension should be only installed on the main wiki. It will be affecting all wikis with the exact same configuration. Rating limiting is applied per wiki, not globally. Administrators are not affected by the limitations.

The configuration of the rate limiters is available through the administration of the main wiki. 

adminmenu.png

You can configure any number of limiters, but usually tree is more than enough. You should be careful that all limiters are applied simultaneously. Therefore, this is not meaningful to have larger time periods allowing lower number of requests since those will implicitly reduce the allowance of the smaller period, which will never really be used.

config.png

The configuration also allows configuration of mail alerts. For mail alerts to work properly, the mailing feature should be properly setup. Emails are sent to the site administrator by default and only when abuse happens. If a user repeat the same abuse several time during a default period of 2 hours, only one email will be sent. Mails could be sent in english or french, and will default to the language of the wiki. All these parameter could be adjusted by the above configuration.

All abuses are also logged to the permanent directory in a log file named: logs/ratelimiter.log. This file will grows, and you should care about cleaning/rotating it to your taste. For convenience, 50 lines of the tails of the logs can be displayed from the administration in reverse order (most recent first). Here is a sample from what the log could contain:

2016-11-24 19:25:56,428 WARN  - User [xwiki:XWiki.user] tried to abuse of [xwiki] by exceeding the 250 requests / 15 min(s) limit. [http://host.example.com/xwiki/bin/view/Main/WebHome]
2016-11-24 17:32:45,213 WARN  - User [xwiki:XWiki.anotherUser] tried to abuse of [xwiki] by exceeding the 250 requests / 15 min(s) limit. [http://host.example.com/xwiki/bin/skin/resources/uicomponents/widgets/tree.min.js]
2016-11-24 16:48:37,179 WARN  - User [192.168.123.45] tried to abuse of [xwiki] by exceeding the 250 requests / 15 minute(s) limit. [http://host.example.com/xwiki/xwiki/get/TourCode/TourJson?xpage=plain&outputSyntax=plain&tourDoc=Main.WebHome]
2016-11-24 15:43:39,123 WARN  - User [xwiki:XWiki.user] tried to abuse of [test] by exceeding the 500 requests / 8 hour(s) limit. [http://host.example.com/xwiki/wiki/test/view/Main/WebHome]
2016-11-24 13:14:02,005 WARN  - User [test:XWiki.testuser] tried to abuse of [test] by exceeding the 250 requests / 15 minute(s) limit. [http://host.example.com/xwiki/wiki/test/get/Main/WebHome?xpage=xpart&vm=commentsinline.vm]

As you can see:

  • the unauthenticated users are treated based on their IPs.
  • local wiki user are supported as well as local wiki access by local/global users.
  • the URL provided is the one that cause the exhaustion, but as you see, it could be some technical page like a JS, of ajax request for comments or the tour application.

Prerequisites & Installation Instructions

We recommend using the Extension Manager to install this extension (Make sure that the text "Installable with the Extension Manager" is displayed at the top right location on this page to know if this extension can be installed with the Extension Manager). Note that installing Extensions when being offline is currently not supported and you'd need to use some complex manual method.

You can also use the following manual method, which is useful if this extension cannot be installed with the Extension Manager or if you're using an old version of XWiki that doesn't have the Extension Manager:

  1. Log in the wiki with a user having Administration rights
  2. Go to the Administration page and select the Import category
  3. Follow the on-screen instructions to upload the downloaded XAR
  4. Click on the uploaded XAR and follow the instructions
  5. You'll also need to install all dependent Extensions that are not already installed in your wiki

Installing with the Extension Manager is highly recommended. If you can't, apart from importing the XAR of this extension, you will have to also install the jars corresponding to the used version in your WEB-INF/lib folder:

Release Notes

v1.0.1

  • Closed RATELIM-2 Mail notification configuration is not translated in french
  • Closed RATELIM-1 Mail notification configuration submit does not save the values

Dependencies

Dependencies for this extension (org.xwiki.contrib.ratelimiter:application-ratelimiter-ui 1.0.1):

Tags:
Created by Denis Gervalle on 2016/11/28 11:50
    

Get Connected