is caching using a webserver's cache (nginx) useful with elgg?

i notice that the elgg simple cache and system cache focus on the static files and elgg engine.
am i likely to be able to improve the site's performance by using the nginx (or other) cache to further refine the caching?
i'm looking at the info on this page and wondering whether to bother or not:
http://seravo.fi/2013/optimizing-web-server-performance-with-nginx-and-php

i'm using apc opcode cache for php..

presently i am seeing the cpu maxing out even with very low traffic on a VPS.. the RAM does not max out or the hard drive.. the CPU reaches 100% on nearly every page visit! as yet i am unsure why..  i think the cpu is about 2.4 GHz.

any tips?
thanks

  • just found what appears to be a useful tool for server profiling:
    www.newrelic.com

     

  • All I can say is that if you can find a way to cache dynamically generated pages it can have an amazing impact on server load.  I have varnishd set up and I find it to be well worth it (some things break and it took a while to find workarounds to make it usable).  Without it I would have to pay 3-4 times as much as what I do now for a server and pageloads would slower even so.  I also use apc and memcached.  APC was a big help as was varnish.  Memcached less so.

    I think the reason varnish helps so much when you set it to cache dynamic pages and essentially make them static is because you are exponentially reducing all the dynamic calls to the server.  It reduces hits to apache (or ningx), mysql, php, and also the disk. 

  • thanks for sharing here.
    i have used newrelic now to greatly improve the performance of my server/ site - mostly though identifying flaws in php code of various plugins.

    maybe i will return to varnish at some point... i am presently altering php code to make elgg more compatible with clouflare's caching (images and other files).

    the dynamic caching of varnish is also available through cloudflare in some senses.. though i think possibly only for paid use.. which i don't do.
    the issue with dynamic caching is obviously that some type of trigger has to be activated so that the caching code knows when content has changed - am i right in thinking you needed to add some custom php code to elgg / elgg plugins to get varnish to function effectively?

  • The biggest win (that's easy) is to use a proxy cache. If you're on Apache, just enable mod_file_cache, super easy. Elgg sends correct HTTP headers so you don't have to override any defualts. This will allow multiple users to use the shared cache of making their own requests for thumbnails, icons, CSS/JS.

    As far as the HTML, you can't cache much for logged in users because so much content is user-specific and dynamic. In the future Elgg needs to consider how to seperate these dynamic elements from content that doesn't change. E.g. when drawing an entity listing, the icons and content tend to be identical between users, but the entity menus are custom; maybe we hide entity menus behind button and load them on request.

    E.g. I think implementing #4459 and #4797 will be major positive steps. That's a ton of HTML that just doesn't need to be delivered in every page, and it would make the markup around them more cacheable. Double win.

  • Any chance you can compile your findings into an easily digestible read of do's and don'ts for plugin authors?  I know some of my early plugins aren't nearly as efficient as they should/could be simply because I didn't know then what I know now.  Chances are the plugins I'm writing now aren't as efficient as they should/could be because there's still probably stuff that I'm not yet aware of and a year from now I'll cringe when I look my current code.  Such is the nature of learning, but a fast-track document for new plugin authors interested in the subject might be worthwhile.

    I've just recently started incorporating this plugin into my projects: http://community.elgg.org/plugins/1222696/v1.0/vroom

    For offsetting long-running scripts without having to resort to a queue and cron, it's fantastic!

  • vazco's node.js model appears to granularise the content sufficiently to resolve what you are speaking of steve..

    i am using nginx so am not able to use mod_file_cache, though nginx is a reverse proxy server and has cache settings available by default, as far as i know - i am uncertain of the most appropriate use of nginx with elgg for caching.

  • i have returned to caching again.. 
    i used varnish for a while with nginx which did function ok.. however i now use https only on the site and varnish does not support https. so i removed varnish and am now looking at nginx's built in cache...

    i have seen the nginx cache correctly serve from the cache for a non-elgg static file; however, the elgg files are all being served non-cached consistently.

    when i examine the page output using curl's verbose mode - i see that the various caching values are not being set for the elgg pages.

    < HTTP/1.1 200 OK
    * Server nginx is not blacklisted
    < Server: nginx
    < Date: Mon, 30 Dec 2013 00:08:54 GMT
    < Content-Type: text/html; charset=UTF-8
    < Transfer-Encoding: chunked
    < Connection: keep-alive
    < Keep-Alive: timeout=10
    < Vary: Accept-Encoding
    < Set-Cookie: Elgg=m3fij64u5k1ccpl4456cpcf812; path=/
    < Expires: Thu, 19 Nov 1981 08:52:00 GMT
    < Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
    < Pragma: no-cache
    < X-Frame-Options: SAMEORIGIN
    < X-Cache: MISS

    anyone know why this is? how do i customise the caching values for elgg's dynamic pages?

  • Caching dynamic pages with a static page caching service is a bad idea. Memcache, APC or OPC are there for improving speed on dynamic pages. Memcache actually is caching intelligently, but I always run into trouble with it. Elgg default caching mechanism is also very effectively. There are some good plugins that help improve page speed. I think that using more ajax to load content below the viewport is one thing that could be improved in general. So use nging or apache caching only for static content is my advice.

  • apc/opc (as far as i am aware) improves efficiency of php scripting, and does not store actual html output. the gallicache plugin is an actual optimised html output cache and when it runs many of the pages load < 1 second for me, whereas they may load in 3-5 seconds otherwise.

    i am looking to achieve that type of caching via nginx, so i do not use gallicache (which is not totally debugged yet). there are some pages on my site which i rarely change, such as the about page and other pages which i do not mind caching for the gained performance. e.g. i would cache the homepage for short times, such as maybe 1 hour or so.

    ideally i would like to apply custom cache values per page and even per page item (as the big sites do). however, so far i am not clear on how elgg approaches this and all items are marked to not be cached - as far as i have seen.. maybe this is due to me using nginx and elgg being originally configured for apache?

  • The pages with the highest load times, are those with a lot of database interaction. Gallicache is avoiding that by treating it as a static pages. I also don't use that plugin, however promising. Because the bugs leads to strange site behaviour.

    Memcache should help with that, but again don't get this stable either and it does not seem to skim off page loading time anyway. But don't expect much from nginx or apache caching, unless you are doing the similar aggressive caching gallicache introduces.

    Some tips that really help:

    - Async load java

    - Check plugins for registering java and load them in cache.

    - Make sure that the code is only loaded in the proper view. A lot of plugins mess this up.

    - Minify css and java. Do it yourself, if the developer forgot it.

    - Deflate (default on in apache) Compression does not reduce page load, but surely improves user performance.

    - Use innodb instead of Isam. I wrote an article about it how to do that.

    - Define /tmp as tempfs so Mysql will sort temp tables in memory. There are a lot of temp tables in Elgg since there are so much joins. Offcourse you need enough memory for this to be available to create sufficient temp space.

Performance and Scalability

Performance and Scalability

If you've got a need for speed, this group is for you.