After upgrade 1.5 to 1.6, some, 10%, of users lost their profile photo :(

On test server upgrade to 1.6 went fine. On production we have problem that some - we dont know how many, maybe 10%, lost their avatar/photo!

Strange is that these people still show in the list of latest users, but now with the dummy profile photo and, normally it shows only people with a photo. So, the system still know these people have a photo, but Elgg just can not find it anymore. Further, if user upload a new photo, the old photo shows again instead of the new one! No photo upload is possible...

We lost ideas... anybody ??


Note: this discussion was reposted ... as for some reason, others could not see the discussion...might have been access, but could not change that

  • Just a clarification -- I didn't make this decision alone. I'm talking about what I suggested to the rest of the team as far as delaying the fix until 1.7.

    @Armin - Sorry, if it's not English, Japanese or Spanish I can't understand :-X

    @Malaga and Master Jack - This will not affect login names.

  • Wow....sorry for the multiple postings.  I had posted from my mobile phone and apparently there was a problem :-X

  • Maybe I'm missing the complexity of it all? The datamigration looks quite comprehendable to me, is it not?   We combine the dirs recursively:

    1. a & A -> a
    2. 0..9 -> 0..9
    3. all other chars ->  "_" (underscore)

    If users with other char log in, e.g. old usernames, they still are redirected correctly.

    In .htaccess we can add simple regex to allow to facilitate redirect for the direct linkers. Existing users keep their usernames. We just end up with less directories ( but still enough for unix find data speed).

    So we should be able to migrate easily?  What am I missing?

     

  • Like I've said, the code to do this is not difficult and has been written.  The problem comes with the time it will take to perform a migration on the entire data directory and the scale of the change, which is inappropriate for a bugfix release.

    Again, usernames will not be changed in the new migration.  We will be using a guid-based directory structure that will be independent of usernames.

  • nc, etc datadir @ local sys fast.. but php.. mem leak ;-(

  • Brett - if it will be php scripting only that handles very large data copying, whatever memory leaks, etc php suffers might slow down the process. We had problems before running elgg/php from web browser on large data migrates -> switched to running php cli from bash. I'll try to find some notes/ emails/ egs and mssg to you.

  • clear... but why not run a script upon first access to some kind of data of each user that still needs to have its data moved? Would it then still take too much time? At least it is then spread over time...

  • @Dhrup - Ah yes it can possibly be a lengthy migration.  Fortunately most modern filesystem's move command will simply rewrite the pointers to the data in the index instead of physically changing around where the data is stored.  This means that the bulk of the time will be spent merging the data directories of users.  On a large site that has run each version of Elgg since 1.0, this might be a significant migration.  For sites that started on 1.6, this will be fast.  For sites that started on 1.5, it will just depend on the amount of users with odd characters in their names, though it can potentially be large.

    @Tom - If we base the migration on when the user logs in, until each user logged in your site would have content "blackholes" where data would just not be there because that user logged in to migrate yet.  The migration unfortunately has to be done at once to ensure the integrity of the site as a whole.  Thanks for offering a different view of this problem, though.  I'm always up for any ideas on how to make things easier for admins / users during migrations!

  • @brett, I tried to suggest to run the script upon each *access* of user's data...not login...

    I know, depends on the site usage if it does not blow up very quick ...still...might be worthwhile to ponder over. I understand your black-holes issue but...remember..that is the issue we now have today anyway since 29 days...so what's the difference? ;)