Any feedback (positive or negative) regarding my anti-spam / IP blocker plugin?

It's been almost a month since I've released the http:blacklist plugin: http://community.elgg.org/plugins/1641554/1.8.2/elgg-1819-httpblacklist (just updated today to offer also a working version for Elgg 1.9).

Sadly, there was no response whatsoever since then. So, I really don't know if this plugin has any value at all and if it's worth to improve it (and anyway feedback regarding possible improvements would be appreciated, too).

Is it that the purpose of the plugin is not clear? The http:bl blacklist is a service similar for example to Stopforumspam, i.e. it's a database of IP addresses that were previously used for suspicious actions like posting spam and other malicious things. The http:blacklist plugin checks against the database when the 'register', 'forgotpassword' or 'login' pages are accessed and blocks / redirects the access if the IP address is listed.

I would really like to know if there's any positive effect is noticeable when then plugin is used (e.g. what does the block counter says?). The http:bl blacklist database seems to have at least an equal number of listed IPs compared to the Stopforumspam database, so I wonder if you notice a similar (positive) effect regarding reduction of spam on your site.

  • @ura : You might be right on trusting an external system, which you can not examine. I also dislike that part since I cannot determine if there are any false postives but with the current plugins, we have no choice to either trust them or don't use them at all.

    On the radical different approach I'd like us to give some attention to an interesting class I found that claims to do what you previously proposed before. A learning system on spam behaviour

    http://www.phpclasses.org/package/4236-PHP-Detect-spam-in-text-using-Bayesian-techniques.html

    I did not do any in depth analysis, the only drawback I see now is that is uses a mysql database for the learning part. Not a real deal breaker, but just something I personally would like to avoid. But I'd like your opininons on this one if we can use it.

  • @iionly, I think I made a mistake on judging the plugin at first sight, so time for a test drive ! I do have to figure out if it is not in conflict with existing anti spam plugins but will report that too if it is.

  • Trust in the external service is a necessary requirement. But if you want to stop spammers BEFORE they register at your site you need to rely on an external service in ANY case, because you can't evaluate the credibility of a future user yet by his actions but only check if the IP address and/or username and/or email address have come to attention for spamming ELSEWHERE. The larger the data pool available by the external service the higher the success rate (at least in theory). And additionally, the risk of false positives when using an external service seems minimal to me. First, it's very unlikely that the credentials of an aggressive spammer gets only reported once (and I do expect that there's some weighting done taking into account the number of reports). Also, both the Stopforumspam and the http:bl replies on querying an IP give back information about the likely risk connected to some IP ("threat level", last reported, etc.) which can be used to tune the blocking / filtering.

    A "learning" spam filter that evaluates postings of existing users won't ever be perfect either. And the question is: can it be much of a success without relying on external data either? Such a plugin would have to identify key words or bad links that are likely connected with spam. But you would have allow all the spam to be posted first before the filter could learn anything. Additionally, the list of key words and links will never be static, so the filter can never stop learning and there will never be an end to spam posts on your site, if you don't want to rely on word and link lists provided by some external service. As Gerard said, 90% of the account creation attempts are blocked. So, you possibly have 90% less spam postings on your site even without the need to moderate / automatic filter them. This is also a question of server and admin resources. Keeping as many spammers out will reduce the server load and also reduce the time necessary for the admin to deal with these accounts and the spam they might have posted already.

  • i perceive here that the needed balance between external data/service and direct admin action can be found through using the external service, yet giving the admin the final say in what occurs on the site. e.g. all new profiles could be allowed on the elgg site and the external service can be used to provide the admin with a backend list of flagged profiles, which can then be blocked/banned through the choice of the admin, rather than through the choice of the external service. the admin can elect to block by i.p. email or other ways (as with cloudflare et al) and can add keywords/phrases to use to more accurately pickup profiles to block.

    this way there is no potential for malfeasance on the part of the 3rd party service and if the interface is intelligently designed in the elgg backend, there will be ease of use too.

  • @ura soul true......that really great ideal i agree with you

  • So, instead of blocking the user, let them finish registration and then let admin (re)validate them if they did not pass the spam filters ?

    But it seems like a good idea. Users still need email validation and than those should still be validated (again) by admin before entering the site. It would provide a way to filter out false positives.

  • the exact implementation has space available for experimenting with different options.
    i would let the users finish registration if the elgg admin has not already blocked the ip address or email address specifically (or if the user does not violate some other type of rule, such as region/city etc.). the 3rd party system would then be invoked to check the user's details and if the user is a new one to the elgg site and is known to the anti-spam system to be a threat, then the user/profile will be added to a list in the elgg admin area, to be blocked/banned/allowed.
    the interface would include options to easily add the user's ip address or email to the local block lists.
    so this is a type of combination of the features of all the different anti-spam plugins.