Migrating Site to New Server

Migrating an elgg website to a new server.. ?  Might be not as straight-forward as it seems.
We've seen migrations of site with 50 members cause headaches, a few hundred members ? bigger headaches ! 29,000 members ?? horrendous task ! Add to the situation the fact that your new server uses a different control panel  (Plesk in our case) -- one had better be comfortable with linux line commands and know the structure of the file system that supports apache, php, vhosts. ps: backups still running from yesterday ;-)  why did those pesky kids upload so much data ? lolz ! I will try to do a technical analysis later if there is interest ;-)

  • @To all friends of FBFK and of the Gang :=

    We have started the migration process for FBFK

    We are using FBFT (FBFTeens) [userbase 767] as the preprod testbed to study data volume issues, data transfer speeds, etc -- so that we can preempt real issues with the 29,000 userbase and related data on FBFK.

    Part of the problems that have slowed us down is that the new dedicated server uses Plesk (not CPanel or LXA/Kloxo which we are used to).. so we have had learning curve here. However by little trivial (read that ==> "expert" thx;) background with linux command line and file-system struc, apache.. has helped to understand Plesk's quirks a little more easily..

    FBT was tootally installed rather quickly, but MJ complained his profile icon was missing !! True... I guess I dummkpof kinda forgot to upload the mega-mega sized datadir ? The download for the whole code + data folders took ummmm.. only about **continuous 24-36 hours ;-) I havent even got to the 1GB++ DB yet..;)

    FBFK migration is next on the board... We wanna be sure we have our shortcuts, streamlining, secret  utilities, any speed enhancers ready and enabled and utilized fully to enable a f-a-s-t backup and reload / migrate of this site that seems to have a very very hugh amount of data that need to be transferred.

    I'll keep y'all posted as things develop...

  • I would like to hear if you ran into any troubles with cgi also.  Hostgator does this cgi cache type of thing to route the php from a public_html folder to a public_www folder.  Questions are what other issues have you ran across in the migration other then size and command line

  • why are you downloading the datadir ??? server to server datatransfer is faster than downloading and uploading again i guess.

  • I migrated this community site (22,000 users) to one of our test servers with no problems at all following the documentation Cash and I wrote at http://docs.elgg.org/wiki/DuplicateInstallation

    One thing I did have to change was an entry in the metastrings table for the metadata that corresponded to the filestore::dir_root's value.

  • Drup - what server were you running before you needed to migrate? Was it dedicated or a vps? If vps do you have any idea how many vps accounts were on that server?

    I'm trying to get an idea of how many user accounts it should be possible to have on a dedicated server (I know there are more factors than just account numbers, but nonetheless nice to know...)

  • @Jayadeep I'm still waiting for some good suggestions from you on SW re: Server-Server FTPs beyond the std proxy'ed FTP

    @JimBob (1) Luv yr profile icon ;-) (2) Going from VPS/A2 Host to 1&1/Dedicated By "VPS Accounts" do you mean Domain owners on the VPS box ? That would not be info the Hosting company would tell us;-( If you're looking at Site's user-base - that is right at the top ~29000. If your question is directed at server capacities - that is a very different area of interest. One would most likely have to study many factors.. We are preparing but not quite started. Just checking our backups, and reviewing other things as we see things such as file sizes uploaded by members for before, e.g. some users had uploaded large files and also many many files....;-( very large uncompressed pics..

  • Data size is indeed a thing we consider as a risk ... Our users tend to have big .bmp files... considering to disallow upload of these all together and only accept jpg/gif/png ...

  • Weeellll;-O


    I am still "struggling" with the actual approach I want use for this DB migrate !!!

    Main problem is ==> @ old host we have no more disk space to even do a realistic mysdqldump backup..

    So.. looking at server-to-server ftp using proxy or maybe linux cmnd level ftp w/ proxy or hotdump of files to my PC to patch data and reload to new server...

    I did do some funky php code to read the .MYD files in **binary for patching.. but code is still "experimental"...



  • I've used a method similar to this approach before: http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/

    The magic will be a combination of netcat and tar...I'd stay clear of using PHP to transfer this stuff, as there are already better tools out there.

  • 3 days to go ;-) comparing nc vs rsync... installed both on tgt svr mc, next inst on src mc - to try some smallish files to test perf. both utils seem to to ok via their descs and reviews..