Elgg 1.8-1.9 upgrade problem: 'deleted user' in appearing in most Messages

Hi all,

Inspiration needed...

When moving from Elgg 1.8.20 - 1.9.7, all upgrades appear to work without error. However,  in the messages tool (which still shows messages that were originally site notifications) instead of a linked profile avatar, about 99% of existing messages are shown with the unlinked words 'deleted user' (none of these are actually deleted users) . The few exceptions where the upgrade has worked are all either from friends, or are direct messages. This makes me think that the main (though possibly not the only) culprits might be site notifications generated by groups, but that could be a red herring.

I have tried the upgrade with and without the site notifications plugin turned on.

Because a lot of searching suggests that this doesn't seem to have been an issue for others in the past, I suspect something in one or more of our existing plugins (or an earlier one) might have led to some oddness in the messages. However, I am not sure where to even begin tracking that down.

Has anyone seen this before? Has anyone found a solution or, at least, a cause?

One possible clue, that might be a side-effect: after doing a few things that might generate notifications on the newly upgraded site, running the minute cron leads to the following error:

Fatal error: Call to undefined method ElggRelationship::getContainerGUID() in /usr/libexec/elgg197/engine/classes/Elgg/Notifications/SubscriptionsService.php on line 65

Apparently this error is known and was fixed in 2.x, according to the discussion at https://github.com/Elgg/Elgg/issues/9780

Prior to doing things that would generate site notifications, the cron runs without apparent error.

After upgrading from there to 1.10.6, there's a different fatal error running the same cron:

Fatal error: Elgg\Notifications\NotificationsService::existsDeprecatedNotificationOverride(): The script tried to execute a method or access a property of an incomplete object. Please ensure that the class definition "Elgg_Notifications_Event" of the object you are trying to operate on was loaded _before_ unserialize() gets called or provide a __autoload() function to load the class definition in /home/jond/elgg-dev/2021/1106/engine/classes/Elgg/Notifications/NotificationsService.php on line 430

Does anyone have any ideas about how I might go about fixing this? I *really* want to keep upgrading to at least 2.3, but am falling at the first significant hurdle!

TIA

Jon

 

  • Has anyone seen this before? Has anyone found a solution or, at least, a cause?

    We've never had a problem like that.

    We upgraded our sites from 1.8 to 3.0 and there've been no errors.

    Does anyone have any ideas about how I might go about fixing this? I *really* want to keep upgrading to at least 2.3, but am falling at the first significant hurdle!

    First, I want to remind you of the update procedure.

    Second. I've already a successful Elgg upgrade experience.

    I know that you're an advanced user and you never admit your mistakes.

    But my opinion is that you made an error in the process of updating the site versions.

    Since you've already done this, I can only advise you to manually correct your database

    ¯\_(ツ)_/¯

     

  • Thanks Nikolai

    Being a not very advanced, very fallible, but somewhat experienced computing professional who has been through the Elgg upgrade process several times since 0.9 (though, clearly, not recently!), who has run a few fairly large IT facilities over the past 30 years or so, and who is a full professor of computing, I do understand the potential for error and I've proven my own fallibility enough times to feel very modest about my own capabilities. That said, this has been attempted with very careful reading of the documentation, careful scripting, and several layers of testing. I would  never dream of doing this to a production site until it had been proven to work more than once, on more than one mirror, so I have now been through the procedure 5 times, on two different test servers, amending the script to perfect it, each time with the same outcome.  I've also been working with an extremely experienced, meticulous and smart sysadmin who has double-checked it all, and both of us have worked with Elgg a fair bit before. So it might well be my mistake, and I truly hope it is, because then I could fix it: but what kind of a mistake could it be?

    I'm not going to manually correct the DB  unless it is the only possible alternative - that way madness lies! It may be salient, though, that we are running MariaDB. It has worked perfectly with 1.8.20 (far more reliably than on the previous 5.x MySQL) - but could there be anything in indexes, character sets, etc that changed in 1.9 that might lead to this kind of problem? There are no obvious errors in the logs.

    I'm working in the dark on this, so any clues would be much appreciated: for instance, does anyone know what *should* happen to old site notification messages during the upgrade? Is it correct that old ones should stay in messages, as they were in 1.8 and older, or should they have been migrated to the site notification plugin?

    Jon

     

  • does anyone know what *should* happen to old site notification messages during the upgrade?

    There was only one important change.

    Just to clarify - have you updated plugins?

  • Thanks again for your continued help, Nikolai.

    Yes, all but one (Vroom, on which a couple of 1.9 plugins still depend) of around 100 plugins has been updated. Most are by (or updated by) Matt Beckett, and most of the rest are by Coldtrick or iionly, so they come from people who know what they are doing. There are a few plugins from less experienced developers but, as far as I can tell, none of those provide or affect content types, or do anything with notifications at all: they just affect the interface. We have a catch-all plugin (again, mainly written by and updated for 1.9 by Matt) that does mess a bit with many things, including notifications (only to add a line or two to the content), so it tends to be my prime suspect when things go wrong, but disabling it before or after upgrade does not solve the problem. If it is the cause, then it already caused the problem in 1.8.

    The site has been running since 2006 and, in the early 2010s after we moved to 1.x, we tried a lot of plugins over the course of a few years that we abandoned or replaced, some of which have left relics of themselves in the DB. There are also those like the Poll plugin, that has been through at least 3 significantly different versions from different developers, that have left stray bits of orphan data in the upgrade process. I don't *think* they should be harmful - they've not caused problems in 1.8 beyond a tiny number of missing posts from pre-2013, and I see no logical reason they should do so in 1.9 - but I have eliminated a lot of other options so the improbable begins to look possible. The fact that this has not (apparently) been seen by anyone else makes me strongly suspect it to be the result of one or more plugins doing something unusual.

    I'm intrigued, though, by the upgrade history you shared in your first reply. Why did you always go to x.x.0 before moving to  x.x.[lastVersion]? That's something I've not tried, and neither the docs nor other advice I've seen in discussions say it is necessary.

    I'd still like to know what is supposed to happen to existing notification messages during the site upgrade: whether they are meant to stay in messages or move to site notifications. Also, I don't know enough about how they are stored, and in particular whether there is anything about them that would identify them as notifications as opposed to other message types. If there were, then I could at least write a plugin that moved them out of the message inbox, and might be able to render them harmless in the process.

    From the error, it appears to me that the messages in question might not be objects of the type \ElggEntity, which would explain why they have no owner, even though they do have content (the messages themselves are displayed intact) and were clearly associated with someone in 1.8 (maybe through \ElggRelationship?). The 'deleted user' message comes from mod⁩/messages⁩/views⁩/default⁩/ ⁨object⁩/messages.php and is triggered if there is no fromId in the message object that has already been identified by its toId.  I have a very vague recollection that, at some point in the distant past, we had to do something to allow non-friends to send messages to anyone (a very common use-case on our site), but I cannot for the life of me recall what it was or why it was needed. Was it/is it that messages is not meant to work between unconnected users?

    My best idea for a workaround at the moment is to write a plugin that trawls through messages, identifying those without a fromId and either giving them one (no idea how yet, but it must be possible or 1.8 would not do it) or, worst case, getting rid of them altogether. If necessary, I will do that, but it would be 100% better to stop it happening in the first place!

    Jon

     

  • Why did you always go to x.x.0 before moving to  x.x.[lastVersion]? That's something I've not tried, and neither the docs nor other advice I've seen in discussions say it is necessary.

    Please learn this: 1, 2, 34, 5

    In addition, we've a lot of experience with such updates and it's the order I mentioned that works best.

    I'd still like to know what is supposed to happen to existing notification messages during the site upgrade: whether they are meant to stay in messages or move to site notifications.

    You may be interested in this: 1, 2, 3, 4, 5

    Also, I don't know enough about how they are stored, and in particular whether there is anything about them that would identify them as notifications as opposed to other message types. 

    Elgg 1.8 uses a function register_notification_object() to register an entity type and subtype to be eligible for notifications.

    In Elgg 1.9, site notifications are separate objects: 

    type: object
    subtype: site_notification
    class: SiteNotification
    
    

    Hope this helps you.

    Good luck!

  • Thanks again Nicolai

    I've read those upgrade docs many times! I'm still puzzled about why you would want to use a potentially buggy minor release to do the upgrade than a patched one with all the fixes, but I gave it a try just in case a new bug had popped up in the more recent version. Alas, it made no difference.

    Thanks for the rest too - useful to have this in one place.

    I have figured out which notifications cause the problem: it's entirely coming from group notifications. In Elgg 1.8 the messages show a group icon but 1.9 needs an explicit sender. That gives me a clue about how to write a plugin to migrate/remove them, but it would still be good not to have the problem in the first place. Again, if anyone knows or can think of a way around this, please share!

    Jon