analysis of elgg performance using the newrelic online app

after some exploration of the extensive server monitoring and performance analysis app - www.newrelic.com

i have found the following with my elgg install (1.8.16):

  • minify: the minify plugin's minification process was being triggered in places where it didn't need to be and was hugely slowing down the website. solution/workaround: disabled the plugin and now use cloudflare's inline minification..
  • language translation: where specific language strings are missing from translation files a helper process of some kind is run to locate a suitable string (i don't know much more than that presently) - which was slowing the performance. temporary solution: disabled all non english languages until this can be resolved more effectively.
  • 'your groups' topbar menu: i am running a version of the plugin which adds a dropdown to the topbar to allow access to all the groups that you are a member of. this is very slow to process the required sql.. solution: find a more efficient query structure (details below.)
  • videolist thumbnails:the php file for the videolist thumbnails is the slowest non core file on my site by quite a long way. the server trace shows that the entire elgg setup of all plugin initialisation processes is being run each time the thumbnail.php file is run.. this is hugely inneficient. solution: the thumbnail php file access process needs to be redesigned.
  • pagehandler.php:some traces for pagehandler.php list the plugin initialisation process as running for 12 seconds per page load! 4 seconds for elgg_load_plugins and 8 for the various plugin initialisation processes. i do have 40+ plugins on my site, however, i think that there is a design flaw in the process here.. why are these plugins reinitialised over and over on each page load? and why are they ALL being loaded regardless of whether they are needed on the current page? surely there is much scope here for improvement? next step: pass to core team for comments.
  • elgg_view: the repetition of the elgg_view function so many times on some pages appears to add an overhead that amounts to potentially seconds of delay.. i haven't done in depth analysis of this, however, some pages (such as the activity river) trigger this view 30+ times per view - this soon mounts up. question:has anyone performed any detailed efficiency tweaking of this?

 

re: the your_groups query.. the present query used is:

    $content = elgg_get_entities_from_relationship(array(
        'type' => 'group',
        'relationship' => 'member',
        'relationship_guid' => elgg_get_logged_in_user_guid(),
        'inverse_relationship' => false,
        'limit' => 0,
    ));

does a more efficient query exist here that can get the current user's groups? can this be cached in a way that it isn't already?

any comment and answers welcome!

  • We should work out some unified benchmarking scheme. There's main problem with many of such tests, that they're hard to reproduce.

    One thing is to have the same data in DB, Elgg behaves differently for different amounts of different objects. I've made a plugin for content generation, but we should have public set of SQL dumps for every Elgg version so anyone could run tests on the same data. Ticket for that: https://github.com/Elgg/Elgg/issues/5765

    Second thing is that running Elgg on cold cache (DB query cache, opcode caches etc.) is way slower than on hot.

    One of most important things is how to evaluate performance. IMO duration of single request is very poor method - it doesn't mean much, it may mean that your disk reads are slow (especially on some VPSes). I wrote more about focusing on server throughput here: http://vazco.eu/news/view/562412

    It's very good measurement to run profiling (like xDebug or xhprof) not only on single requests but during high concurrency benchmark. Propotrions differ a lot.

    From my experience there are currently two main problems in Elgg, one is using DISTINCT in SQL (ticket for that https://github.com/Elgg/Elgg/issues/4594) and lack of view-level caching (that's tricky to achieve, but should give huge benefit for stuff like menu system). I didn't notice performance problems in elgg_view function itself. Would gladly look at some insightful data.

    According to initialization performance. You can ruin it if you use plugins that are poorly written and if smb does ugly stuff on system init event, that can't be fixed in the core. This event fires for every plugin by definition. From my banchmarks, the registration itself is negligible, so it's mostly the matter of 3rd party plugins quality.

  • @tunist I don't like your idea about ALL minification&caching in Elgg (just wait 1.9 release)

    But thanks for your experience :)

     

    About List of User's Groups.

    We're using this for 'page_menu' section:

    $owner = elgg_get_page_owner_entity();

    if (elgg_is_logged_in()) {

    $user = elgg_get_logged_in_user_entity();

    if ($owner->guid == $user->guid) {

    $groups = $user->getGroups('', 6);
                foreach ($groups as $group) {
                     elgg_register_menu_item('page', array(
                        'section' => 'groups',
                        'name' => "group-$group->guid",
                        'text' => $group->name,
                        'href' => $group->getURL(),
                    ));
                }

       }

    }

     

     

  • @RvR Did you do by chance comparison of the amount of "using temporary" and "using filesort" before and after introducing it?

  • i agree that a shared benchmarking approach would benefit us all, for sure.. at least shared data, yes.

    i appreciate that caching is different on each server, however, the points i identified are going to be consistent across hardware because they are php flow / database flow based.. the efficiency of this code is 'standalone' in the sense that great caching will only mask the underlying design weaknesses.

    the software i am using can show whatever type of analysis of performance you prefer.. over 30 mins.. or many months.. you can then view specific traces of specific page loads, so i am not just picking up single pages with 'freak' load speeds that may be due to extraneous phenomena (that's the 1st time i have used those 2 words together.. lol).

    elgg_view: i just scanned the code for elgg_view and can see that there are 15+ IF statements before the view is executed.. which means that for some pages there will be 400+ if statements executed within elgg_view as opposed to potentiall zero being executed if elgg_view didn't exist and each view fully coded its own content in a non modular way..
    i appreciate the value of a modular topology for the php, however, i think this particular function has considerable need to be simplified.


    here's a partial trace of the group profile page on my site loading very slowly.. (i am using widget manager to render various list of items on that page and thus many elgg_views occur).

    image

  • aha. 'getGroups'.. ;)
    thanks

  • @Paweł Sroka Nope. I've missed this step :(  But I've seem that before using ALL rows was taken from the table. And now selected a few rows without the filesorts and temporary scan ;)

  • @ura Note that times on your screenshots are incluseve, so it shows time it took to execute particular elgg_view call. I'm pretty sure it's mostly time of evaluation of include(view_file_path) (https://github.com/Elgg/Elgg/blob/1.8//engine/lib/views.php#L501). I hardly believe that 400 simple if statements that must be done anyway, would do that big difference. Running trigger plugin hook, that has several ifs, ~2500 times sticks around 5ms total execution time on my test machines.

    It's not supprising that content generation takes time and it's the matter of optimizing (or caching) it. Removing few ifs is unlikely to make much difference. Of course feel free to prove me wrong :-)

    Just saying, it's not worth the effort to focus on microoptimizations.

  • @RvR Tough luck. I'm glad to hear it works fine for you :-)

    I want to remove at some point all unefficient calls from the core and give an API option to use DISTINCT only when really needed. Right now we use uneffective modifier to cover design flaws when constructing egef_* calls.

Performance and Scalability

Performance and Scalability

If you've got a need for speed, this group is for you.