analysis of elgg performance using the newrelic online app

after some exploration of the extensive server monitoring and performance analysis app - www.newrelic.com

i have found the following with my elgg install (1.8.16):

  • minify: the minify plugin's minification process was being triggered in places where it didn't need to be and was hugely slowing down the website. solution/workaround: disabled the plugin and now use cloudflare's inline minification..
  • language translation: where specific language strings are missing from translation files a helper process of some kind is run to locate a suitable string (i don't know much more than that presently) - which was slowing the performance. temporary solution: disabled all non english languages until this can be resolved more effectively.
  • 'your groups' topbar menu: i am running a version of the plugin which adds a dropdown to the topbar to allow access to all the groups that you are a member of. this is very slow to process the required sql.. solution: find a more efficient query structure (details below.)
  • videolist thumbnails:the php file for the videolist thumbnails is the slowest non core file on my site by quite a long way. the server trace shows that the entire elgg setup of all plugin initialisation processes is being run each time the thumbnail.php file is run.. this is hugely inneficient. solution: the thumbnail php file access process needs to be redesigned.
  • pagehandler.php:some traces for pagehandler.php list the plugin initialisation process as running for 12 seconds per page load! 4 seconds for elgg_load_plugins and 8 for the various plugin initialisation processes. i do have 40+ plugins on my site, however, i think that there is a design flaw in the process here.. why are these plugins reinitialised over and over on each page load? and why are they ALL being loaded regardless of whether they are needed on the current page? surely there is much scope here for improvement? next step: pass to core team for comments.
  • elgg_view: the repetition of the elgg_view function so many times on some pages appears to add an overhead that amounts to potentially seconds of delay.. i haven't done in depth analysis of this, however, some pages (such as the activity river) trigger this view 30+ times per view - this soon mounts up. question:has anyone performed any detailed efficiency tweaking of this?

 

re: the your_groups query.. the present query used is:

    $content = elgg_get_entities_from_relationship(array(
        'type' => 'group',
        'relationship' => 'member',
        'relationship_guid' => elgg_get_logged_in_user_guid(),
        'inverse_relationship' => false,
        'limit' => 0,
    ));

does a more efficient query exist here that can get the current user's groups? can this be cached in a way that it isn't already?

any comment and answers welcome!

  • We're in a process of preparing presentation regarding Elgg and performance. You can already find two first parts on links below, probably they will be usefull:

    Elgg and performance - basics
    Elgg and performance - for developers

    Ura, probably a part about optimization near an end of the first presentation will be most usefull for you.

    Third part, prepared by Paweł Sroka, is just rendering, we'll upload it tomorrow.

  • yes, thanks for the videos.. :)

    the 'remove heavy features' part i have done - with the exception that i need to recode the related_items plugin which is inefficient presently.

    i am using apc and cloudflare caching - though i don't have much data yet on how effective that is with the current configuration.. i don't have any custom apc calls setup.. i haven't seen much comment on how to do that effectively in practise - so for now apc is doing whatever apc does by default ;)

    the other aspects mentioned such as node.js and client rendering vs. server rendering are interesting and make sense.. except i am not sure of the details enough to know, for example, whether this might be a problem for low cpu rated clients. i am not very likely to go that much in depth into recoding elgg at this stage though.. if elgg core is changed to include this approach i would gladly use it, for sure.

  • i found that blitz.io does not allow sites with pages larger than 300KB and currently, even with gzip activated on nginx, the main pages on my site are between 450KB and 500KB!
    so i used loadimpact.com to do a similar test which doesn't have a page limit (that i am breaking).. i will continue to explore there.

  • looking at videolist's thumbnail.php page i see the line:
    require_once(dirname(dirname(dirname(__FILE__))) . "/engine/start.php");

    if i comment out the line then the thumbnails do not render..
    what is the appropriate way to restructure this page so that the engine is not explicitly loaded for the thumbnails as if they were a seperate page?
    i am imagining the answer is to move the page to a view. perhaps the plugin is this way because it was created in an old version of elgg and never restructured as elgg evolved?

  • oh so.. digging deeper..
    the videolist icon process is quite convoluted.

    the videolist icon view already exists and calls getIconURL which feeds to the icon override hook which then calls the videolist thumbnail.php page which then initialises all the plugins as if an entirely new page is being rendered.. which then returns a thumbnail via a multi line 'header' push (if push is the right term).

    so while steve's idea of caching the plugins via apc would be helpful (though i had assumed that the 'system cache' in elgg already did this) - the basic concept of treating icons as their own pages and pages as needing duplication of the entire elgg engine seems to be highly inneficient to me at this point.

  • @ura soul , you can avoid engine to load for thumbnail.php , you need to register a new page in your plugin and put contents of thumbnail.php in that page file.

  • how does that method change whether or not the engine is loaded?
    that method appears to me to be a repeating of the same model that already exists and actually adding further complications, not simplification. or did i miss a detail that was not written!? ;)

  • re: engine loading:

    i searched through the mod directory for the line that includes the engine/start.php file and found about 10 plugins that use that line (on my installation). one of them is the profile plugin which has it in there twice!
    what the heck is the purpose of that?

  • If you look at the profile plugin, it actually handles user icons without booting the elgg engine and is the most efficient.  That line in there is the fallback in case something goes wrong, then it boots the elgg engine.

Performance and Scalability

Performance and Scalability

If you've got a need for speed, this group is for you.