Light Load of the Engine?

Has anyone considered enabling a light load of the engine for pagehandlers that don't need access to other plugins? Or is there a way to do this I haven't discovered?

I was able to improve pageload times by up to 400% by disabling plugin loading for pagehandlers that were only redirecting to image resources. I set a flag right before including engine/start.php, and hacked that page to skip plugin loading if the flag was set.

Specifically, I disabled plugin loading for:

/mod/groups/graphics/icon.php **for this one I also created a mod rewrite rule to bypass the pagehandler so it doesn't need the overhead of loading the groups plugin just to load a 'tiny' thumbnail.

Is there a better way to do this?

  • @Z

    interesting to see renewed interest in these areas. Oliver's Compactor (314 sql queries;-) and other utilities most probably should work unless the code will require API call 'upgrading' for v.1.7.x

    i would be most curious to see your actual code and techniques used for the 400% speed improvement. i think people who run elgg-bases sites with very high traffic might also be interested in your research in this area and results.

    the idea of a "light" engine as you decribe has been broached before, though i do think there was much serious discussion in the past - just sporadic comments and then apathy....

  • Haha. Apathy tends to sneak sometimes.

    My hack is really simple:

    Line 112: if (($installed) && ($db_installed) && ($sanitised) && (!$fast)) {

    I just added the (!$fast) condition.

    Then, right before I include the engine, I define that flag.

    Line 72: $fast = 1;
    Line 73: require_once(dirname(dirname(dirname(__FILE__))) . "/engine/start.php");

    I should mention that this is just for images that aren't cached. So it has bigger gains on plugins that don't even implement simplecache (like groups and form)  There are probably other handlers I haven't found yet that I can add it to.

    I plan on implementing memcache before my site goes live, so maybe this wont be an issue. But it still seems like a good idea to have a lighter engine for certain pages. Or maybe  break plugin loading out into a separate engine include?


  • ;-)

    "... break plugin loading out into a separate engine include?...."

    Yesss !

    Even for most page loads - it would seem that not all of the code for all plugins really need to be loaded at all. Some pages may require widgets and it is unfortunate that plugins that also have their own meny items and their widgets define these inside their code - and so we need those particular plugins to be loaded. Maybe if plugins proper and widgets had been demarcated - We could then define which plugins and which widgets need to be loaded --> Could end up with a lighter and faster loading engine.

  • I guess all systems reach a point where extensions create too much overhead. I'll think on this and post again if something comes up.

  • A few comments:

    • mod/profile/icondirect.php bypasses the Elgg engine if simplecache is enabled.  That's the point of it.  If you're getting to line 73 simplecache is either off, messed up, or we have a bug that's preventing it from being used.  (Checked in 1.7.4 and it correctly bypasses the engine for me.)
    • Oliver's original benchmarks were on Elgg 1.1 or so. Huge improvements were made in caching for Elgg 1.5.  On a fresh installation of 1.7.4b, loading the dashboard and all its resources has 54 queries.
    Simple cache attempted to solve the problem you're describing, and actually does on the few pages it's used.  The problem is that it's fairly cumbersome and inflexible.  We're always looking to improve performance but I'm not convinced a light engine is worth pursuing as I believe there are better solutions with generalized caches.
  • Yeah, I think you're right. Long term, I'll work on caching and other scalable solutions. This was more of a quick fix to speed things up for now. But it is faster to implement than enabling caching for all the plugins that use pagehandlers for lightweight operations.

    I couldn't get Compactor to work in 1.7.3. None of the plugin settings are recognized, so I think it's not being accessed. It does seem to be out of date from the current release so I'm not going to pursue it.

  • 314 queries @ v.1.1 --> 54 @ v1.7.4b .....nice ;-)

    Overall - maybe one should consider the old old saying -- if you to go faster - get a bigger higher powered engine (no pun intended). If too many queries -> just get a better more hardy linux box to house mysqld. Squid, APC, TurckMMcache, etc do improve overall performance. So I guess we can look for code level optimization (always a good idea) or "soup up" the server power.

    Having said all that from both sides of the argument now - I'm still interested in looking at ways to optimize the code itself - as Brett's comment re: v1.7.4 demonstrates - cleaner, leaner, tighter code and etc etc that the Elgg Team has done to make 1.7.4 so much faster..