Redis+Elgg=Awesome

why shouldn't elgg core team have look at http://redis.io , http://mustache.github.com and https://github.com/nrk/predis this would make elgg faster and awsome.
Suggestions are welcome ;-)

  • In the short term Elgg already has limited support for memcache and work needs to be done to synchronize ACLs into the cache to avoid more queries.

    After that we're going to use a cache system that supports multiple and composite backends. Then you could cache to APC and Memcache or Redis or whatever. But don't expect big gains over memcache; we can only use it as a key-value store at this point.

    On the front end were aiming to make Ajax and client rendering easier. Mustache may/may not help us there.

  • Steve, we work with Elgg that uses node.js, REDIS, mustache and Mongo for over a year now. We built three systems with use of this technology. I can really recommend this solution. Mustache is light and in the same time quite powerfull. Key-value caching based on REDIS improves performance from 3 times to 30 times, based on page specifics (with average being approx. 10 times)*. We use Mongo only for more specific features.

    In case you decide to use mustache as a view system for Elgg, I think there's much help we can provide you with.


    _______

    *We also have an option to increase performance even more, although at this stage it's not required for our projects.

  • Steve, just to clarify, as I think we're talking about two things. If you switch memcache to REDIS, this don't increase performance much. We use REDIS to cache mustache views. The biggest advantage here is that most requests don't go to database or PHP, but simply to REDIS, without use of PHP.

  • @mikevazco: is any particular php client more suitable or better for redis + elgg ?
    what's the performance gain% like for the mousache`d views ?

  • Hi guys,

    Well we have created nginx + elgg + mysql (primariy data source) + ElasticSearch cluster(secondary datasource) + Mustache. ElasticSearch has flexible document structure, So it really fit to elgg EAV model. I have experimented over elasticsearch via elgg. Entities pulling is around 10000 / second. And we have imported 400K entities in our database.

    @Vazco: We have tried to cache mustach templates in js files itself. once loaded on the client get cached over there. And than there is even no more call to web server evan.

  • izap, I used bad wording. Of course our mustache views also are cached on client's side in js, this is how mustache should be used. Data for those views is however cached in REDIS. You have to get data from server for dynamic pages, this is where REDIS helps.

     

    Regarding search - Elastic search is quite good solution, however also quite complex. We used simplier solution, at first basing at this idea. It worked for us very well for a website with 160k users and search by 8 parameters. The modifications which will allow to put this richer solution to core Elgg I believe are waiting in github to be accepted.

  • [Moderator: this comment was off-topic. It was moved to its own topic. Read it here.]

Performance and Scalability

Performance and Scalability

If you've got a need for speed, this group is for you.