Posting speed here at this site

Hi, I've noticed over the last couple of days here that while read speed is quite snappy, that posting a comment incurs huge time lags - like tens of seconds.

Is there something going on with the database that is slowing the ability to make posts here?

Just wondering,

doug

  • My suspicion is that it's the foreach through each group member to check the user's personal notification settings.  That's a lot of database calls for get_entity()...

     

  •  Without knowing how your code works, I have what is probably a dumb question. No matter how long it takes to loop through each group member to check personal notification settings, why does that have to be done prior to the response page being generated for the poster's browser?

    Why isn't it done afterwards?

    It doesn't seem like it is something the poster need wait on before seeing their confirmation page, does it?

    In fact, a lot of systems deliberately wait a few minutes after a post before triggering notifications to give a poster time to go in and correct last-minute typing mistakes.

     

    doug

     

  • @Cash - I agree RSS might be better, but I wonder how many people are dependent upon the emails...

    @Doug - What you describe is a queuing system and a wonderful idea.  Elgg's notifications don't work this way because they weren't programmed to :)  The notification system is still moderately new and under-developed and it's apparent some sort of queuing system is required on large sites.

     

  • Hi, Brett.

    The bit about a 5 minute pause before sending notifications would require queuing. But does sending the email notifications after showing the response page to the user require that?

    Can't the user's post be displayed and then the server goes about it's business sending notifications?

    Right now the server can handle multiple requests at the same time. You can have lots of users online all at once. Each user doesn't wait for another user's request to be handled before they can click and look at a page, right? Multiple sessions are handled at once. Isn't it a matter of how the code is organized?

    Right now it sounds like (1) the user clicks submit, (2) the post is created, (3) there is a long loop to send out the notifications and finally (4) the response page is formed and sent back to the user's browser.

    There's no way of switching the order of (3) and (4) and let Elgg take care of the notifications after returning the page?

    How do sites with tens of thousands of members deal with email notifications?

    Thanks,

    doug

     

  • There's no way of switching the order of (3) and (4) and let Elgg take care of the notifications after returning the page?

    Short answer is no.  When you post you are posting to a different URL (an action) than the one that displays the forums (a page handler...at least, what should be a page handler).  It is in the action URL that notifications are made.

    How do sites with tens of thousands of members deal with email notifications?

    They have implemented asynchronous notifications or a queing system.  It's not a technical difficulty, it's just something that wasn't implemented in the notification system because it's still young.

  • Hi, Brett. 

    I won't belabor the point because you are very experienced in the code and I know zero about it. But if you don't mind just one more question...

    When you post to an action, couldn't the code do something like an http redirect to the page handler after the post. And then go on and do its email notifications?

    I guess I don't see why the poster has to be part of the email notifications process that takes place.

    Thanks,

    doug

     

  • Doug is correct. The simplest queuing system would send them out on the shutdown event so it doesn't delay the forward() at the end of the action.

  • Yeup...that's definitely a possibility for queuing.  As I said there is nothing technically difficult in doing this--it's simply the fact that the notifications system wasn't programmed to do it and hasn't been revamped.  Right now the notifications system reacts immediately to the creation of new entities, which is causing these speed problems when posting.

    On a similar topic, I have also noticed that long threads have a long delay between "Save" and actually starting the post event.  I believe this is a bug in how TinyMCE is handled in each thread but haven't been able to look into it yet.  Just putting that out there so everyone knows I'm aware of the double-sided problem :)

  • Would anyone be heartbroken if I disabled email notifications for groups until we can implement a queuing solution?  The wait is absurd...

  • Hi, Brett. Well, I have had greater heartbreak in my life :) but if email notifications are turned off I probably end up not following these groups anymore, even if I wanted to.

    I didn't want to make a point of it when somebody previously mentioned that using RSS subscriptions are better, but I disagree with that opinion for many reason, including:

    1. Everybody gets email. Even my 82 year old mother. For RSS you have to set up and use a separate client. I actually don't have an RSS client set up either on my Mac or on my iPhone. Email is the "universal common denominator" for notifying people.
    2. Instead of being notified of new messages wherever I am, I would have to remember to go back and check groups for new messages. That's why I stopped using Twitter after about a week - there were no email notifications. In fact, what's the advantage of RSS feeds over just going back to the web forums if you have to go back and check anyway?
    So while I won't be devastated and heartbroken if you turn off email notifications, it will mean I probably will not notice new postings anymore, which would be a shame.
    Email notifications are great! :)
    Just one person's view. 
    Thanks,
    doug