optimising elgg database and making custom sql processes

in the process of seeking to optimise the related-items plugin for elgg, i searched google for the phrase that i have used as the title for this thread.

i found this:
http://stackoverflow.com/questions/1564096/how-flexible-is-elgg

which echoes what i have been experiencing and noticed, particularly about the vars and config global variables.

so my 1st question here is: 'does anyone know of any plan to implement a better schema than is active in 1.8 elgg'? I see many changes and comments in github, since i am not present in those conversations, i am unaware of the details of what is being changed.

my 2nd question is: 'does anyone have any knowledge of how to optimise a plugin's database usage in elgg?' i am uncertain how best to add indexes and custom SQL in a way that elgg likes and that will be maintainable across future revisions.

 

thanks

  • Those are comments from people that simply do not understand Elgg, much of what's stated there is incorrect.

    It keeps two global objects ($varsand$CONFIG) that have more than 5000(!) members loaded in memory on each page. This is a crap indicator.


    Possibly due to crappy custom code in the project, but Elgg itself doesn't keep all users in memory by default.

    1: why do you need a new schema?

    2: you can add your own selects, joins, and wheres using the getter functions.  If you absolutely need to query outside of the getter functions you can use one of the low level database functions such as get_data('SELECT blahblahblah'); but keep in mind at that point YOU are responsible for making sure there are no SQL injection issues and honoring things like access permissions.

  • i have not delved deeply into those two variables.. i do know that i am uncertain as to what exactly they are and that often when i dump them using elgg_dump i see vaaaaast amounts of data being dumped.. which is not necessarily an issue at all.. i imagine that $vars is the global set of paramaters for the entire elgg site across all active processes.. yet that is really not much more than a guess.

    what i am seeing really, is that the elgg design's intentions to provide a layer between coder and sql is ok, yet has so far come with an imbalance that the actual details of the sql processes are not documented thoroughly (or openly) enough in english to be so useful to me.

    i don't need a new schema as such; i am just asking if there are plans to do so. my needs are greater transparency and capacity to optimise the system to support functions that are not supported (in an obvious way?) by core elgg.. e.g. intensive data processing/searching on tags as with the related-items plugin. 

    so if i want to add indexing for particular elements - how do i know what the intended indexing structure for elgg is, by design? where are the documents on this? i searched the documentation wiki for indexing and found nothing of note.

  • It keeps two global objects ($vars and $CONFIG) that have more than 5000(!) members loaded in memory on each page. This is a crap indicator.

    I think the "5000 members" part was in reference to 5000 class member, i.e., properties of the $CONFIG object. We know that's bad and there was a good chunk of work in 1.9 to mitigate that.

    The $vars part is completely wrong, though.

    Keep in mind that this question on SO was asked 3 about and 1/2 years ago...

  • ' it's not built for programmers like drupal is.. '
    ohhh lolz & lolz & lolz.. ;oO;X;P
    and they do say that laughter is the best medicine !
    the 'nosql in mysql... ' was designed by those that are
    close to genuises in computing sciences and.. 
    here are a few amateurs that cannot code - 
    posting out the crap from inside their skulls to criticise ?! 
    @uras - your issues with the poor performance of your plugin 
    is a result of your own coding doing...!  
    nothing at all to do with the intrinsic design of the data model which is genius-like in elgg. 
    they also say that a poor workman quarrels with his tools! 
    those pseudo programmers at that cakephp forum are that.. 
    you wanna join them to trash elgg ?;-P go ahead spoil your day!
    i told you the first day you mentioned the plugin's performance issues --
    you've coded all that ~50 lines of php ! lolz code ! instead of *one sql call
    to extract and coalesce the data needed - maybe about 10 lines of code. 
    matt's statememnt is wrong -- ' from people that simply do not understand Elgg '
    i say -- those guys do not understand *programming ;-)

    i'm outta here..
    bye.

     

  • dhrup: get back to me when you have stopped the perpetual quarrel in your own mind

  • uras:
    there was never a quarrel..
    when your plugin came out and you complained about the performance and the OO/M issues;
    i posted some very good technical advice abhout why the 'slowness'
    -> it was and is the massive sql results being 'filtered' - individually by hand (php)
    tha's causing the certainty of the problem
    ~~> you choice = reply with whatever 'smart' words...
    OR
    investigate into how the sql elgg api call can be re-written properly
    without extra indexes, etc..
    .. that is your code-bridge to cross and re-code by yourself
    -- take the best that has been given and 'run with it' and quarrel with the issues ;)
    but no amount of ' add indexes and custom SQL.. '
    will fix the 'perpetual' problem that can be solved by the elgg api alone..

    i have coded pure elgg sql call to get same results without the excessive php..
    -- merely for my technical curiosiity ;-oO
    you will have to cross your learning bridge for yourself by yourself
    so that you may learn to do these things youirself after
    good technical advice has been given already..

    ' it is when the fight begins within that a man is worth something.. '

     

  • 1. i did not complain about the performance and oom issues. they were reported to me and i have sought the remedy. you are inventing and projecting; how creative of you!

    2. "the massive sql results being 'filtered' - individually by hand (php) tha's causing the certainty of the problem" -> yes, i know. regardless of what post processing is or is not coded, if the database is not configured to support efficient access to the data that my code requires then the code will not be efficient. hence a need to know about indexing and thus the search for schematics and the not finding of schematics (or any text at all on the subject) and hence the enquiry here.

    this is a simple topic if you are not continuing to place your own worth on combat / sport and other such denials and delusions which obscure that actually you have self worth whether you can beat someone at something or not. in truth, claiming that worth needs to be earned is part of why you appear to not have self worth and why you desire to create self worth.

    ' it is when the fight begins within that a man is worth something.. '
     
    ->
    'you are infinitely worthy and believing you can increase your worth through fighting will only cause further dysfunction and compromise whatever worth you are allowing to manifest as greatness in yourself; repelling those who know they are worthy from you'.