Migrating Site to New Server

Migrating an elgg website to a new server.. ?  Might be not as straight-forward as it seems.
We've seen migrations of site with 50 members cause headaches, a few hundred members ? bigger headaches ! 29,000 members ?? horrendous task ! Add to the situation the fact that your new server uses a different control panel  (Plesk in our case) -- one had better be comfortable with linux line commands and know the structure of the file system that supports apache, php, vhosts. ps: backups still running from yesterday ;-)  why did those pesky kids upload so much data ? lolz ! I will try to do a technical analysis later if there is interest ;-)

  • If you don't care about getting a raw SQL dump and just want to copy the actual files and directories MySQL makes, rsync is the better choice.  If you want to create a tarball of the mysql-dump output, nc is the way to go.

    They are two very different programs, but both very good at what they do.  Depending upon how you want your data migrated, either is a good choice.

  • ;-) thx for yr input..  i have been sort of leaning towards xfer of FRM MYD MYI files to new bigger server where there's better resources ( mem, disk, etc..).. can still edit/ zap ;-) those files for old/new values ;-)

    ps: i did do a cpl of tests w/ phpmyadmin on our bigger tbls b4... killed the server;( *that's when i gave up the $5 host mentality and small DB sizes and crawled back to cmd line utils..

  • @dhrup - I'm not sure about this but I wonder if mysql replication can be used to synch up the old and new databases and when fully synched, switch over to the new site after all the uploaded files have also been synched?

  • @Alex
    ;-(
    We never did have mysql replication from the old server to the (very new) server...
    so... looking to "cheat" and do some trix to xfer with very little (critical) time span...
    now.. maybe i've not looked too deeply at replication from old ("beaten down") server...
    but.. old server is too pushed for resources - disk space running out ;-( cpu memory limit causes problms... (sobs..)

  • oki ;) thx for the ptrs..
    did mickey-mouse test netcat'd an 8K FRM => abt 0.01 sec to xfer ;)
    next to netcat xfer largest file in test db abt 2GB to see speed, high expectations ;)
    then rsync on same files to compare...

  •                TEST SITE   LIVE SITE
    DATA    138 M            13 G
    MYSQL     9 M             1.1 G

    SRC     ==> grab data -> tar -> gzip -> send
    TGT     ==> receive -> gunzip -> untar -> save data

    getting ready..;)

     

  • Tested RSync speeds and features as well ;-)

    approx 150 MB in about 5 mins..

    @Alex:

    re: the mysql replication --

    Apparently RSync has tricks to allow such a "replication" by copying the MYI MYD FRM data stores similar to mysqlhotcopy (for MyISAM types which is what defaults for Elgg DB). The slight advantage here is that RSync can xfer the DB data stores even while MySql server is running -- then at the last moment only does one need to flush, lock and and resync for the incremental DB *changes only since the previous RSync.

    Plan now is to run staggered RSync uploads until dday, then shutdown for the mimimum time and edit/ patch/ restart.

     

  • http://elggtest.ensci.us/blondie.jpg
    seems we have the same problem that some others have reported ;-(
    we done some investigating and...
    the fix for this when migrating up to v1.6.X
    we believe.. will be *not trivial code...
    leeenoox does allow for almost *any character in filename LOLZ and then forever afterwards  *itself cannot retrieve the same LOLZ x 2... major quandry we have...

  • whew ;-) seems tinymce writes "some" html hard code thingies using the old server's subdir name.. more to sed zap..;-(

    # grep -i -c 'newsite/' *.mysql
    elgg_datalists.mysql:1
    elgg_groups_entity.mysql:1
    elgg_metastrings.mysql:122
    elgg_objects_entity.mysql:33

  • the following are plugins that I have hade problems with when moving site dirs' and or the site domain name

    poll Just didn't work

    Jeromes custom style (lost the ability to upload your own background)

    tinymce also all pictures that where not profile pics ????? (however this is not a standard tinymce)

     

    there are also other plugins but not sure now it's been a long time