Uploading Large Files

You have a new message from harcha24. It reads:
From readin your profile I saw that you were way into elgg and kinda into
the bass.  I'm the opposite.  I just got started with elgg but have been
a music instructor for years.  I was hoping to pick your brain for a
second.  I am attempting to use my elgg for musician purposes but can't
seem to get elgg to allow me to upload large files (very large).   I
tried changing the .htaccess file but it dosen't seem to change anything.
could you help?
also I would be interested in any ideas you've come across to implament
more music file control in elgg or other application?

-------------------------------------------------------

  • (1)    Please do not message me asking for help
  • (2)    Post requests for help to the group here.

File upload size is controlled from php.ini

 

 

 

  • u r welcome.

    It is better to post openly in a group so that everyone can benefit from the answer ;-)

  • You can increase the file size limits in your php.ini, but you may then encounter other issues.

    I'm getting out of memory errrors for anything bigger than 25meg on my sites, which are on shared hosting, therefore I can't test a big file. 

    It would be cool if the file upload code could split large files into chunks.

  • one of my elgg sites is a recording community where people would need to uplaod very large (probably zipped) files.  by very large i mean like 500mb.  maybe there is a better way if anybody has any ideas.

  • you got a *big problem..

    out-of-memory > 25 MB means we start increasing memory if thqatwill make upload go ok but.. at some point in time - your host esp the $5 - $20 / month pkgs will either have a talk with you or kick you out ;-( if your hosting is more powerful, maybe dedicated @ $100++ you'll have better luck.

    500 MB is very large piece of file !!

    Q ? how come your files are s-o-o-o- b-i-g ?

    are you talking about very high quality audio ? wow !!

    assuming you host will allow you store such large files... - difficult...may even timie out on the upload.

    maybe able to code some kind of utlity to upload many smaller files - file001.. file200 @ 1 MB each and then have code on the server re-join those files, this means user has to be able to split the files first. but now then we're getting into some heavy coding -- if this is the way to make it work ;-(

    if you *can upload a much smaller zip or tgz  of the original music file.. it will be not so difficult to unzip or untar etc on the server, but you need to know your php and linux stuff to do this.

  • 1 file = 500mb uncompressed CD or a video of some kind ???? 48K stereo audio runs at approximately 10mb a minute

    12 songs* 4minutes = 480mb 

    Sound fonts can run to 500mb as well

  • Here is my Host's help file on this topic. Everyone looking for larger file uploads might just want to read your Hosts help files too because it really is not an Elgg issue ;-)

    Knowledgebase

    How to optimize your PHP installation to handle large file uploads.

    Problem:

    How to optimize your PHP installation to handle large file uploads.

    Solution:

    Follow this guide and place the php.ini in whatever directory the php file is being executed from.

    Howto optimize your PHP installation to handle large file uploads.

    Though PHP presents a very versatile and user friendly interface for handling file uploads,
    the default installation is not geared for working with files in excess of 2 Mega Bytes. This
    article is an aid to configuring PHP for handling suc h large file transfers.

    php.ini

    The php.ini file contains all the configuration settings for your installation. Sometimes
    these setting might be overridden by directives in apache .htaccess files or even with in the scripts themselves but for the moment let's just concentrate on the ini file.

    This file contains the following settings that we need to modify

    • file_uploads
    • upload_max_filesize
    • max_input_time
    • memory_limit
    • max_execution_time
    • post_max_size

    The first one is fairly obvious if you set this off, uploading is disabled for your installation. We will cover the rest of the configuration settings in detail below.

    Files are usually POSTed to the webserver in a format known as 'multipart/form-data'. The post_max_size sets the upper limit on the amount of data that a script can accept in this manner. Ideally this value should be larger than the value that you set for upload_max_filesize.

    It's important to realize that
    upload_max_filesize is the sum of the sizes of all the files that you are uploading. post_max_size is the upload_max_filesize plus the sum of the lengths of all the other fields in the form plus any mime headers that the encoder might include. Since these fields are typically small you can often approximate the upload max size to the post max size.

    According to the PHP documentation you can set a MAX_UPLOAD_LIMIT in your HTML form to suggest a limit to the browser. Our understanding is that browsers totally ignore this directive and the only solution that can impose such a client side restriction is Rad Upload Applet

    When the PHP engine is handling an incoming POST it needs to keep some of the incoming
    data in memory. This directive has any effect only if you have used the
    --enable-memory-limit option at configuration time.
    Setting too high a value can be very dangerous because if several uploads are being handled concurrently all available memory will be used up and other unrelated scripts that consume a lot of memory might effect the whole server as well.
    These settings define the maximum life time of the script and the time that the script should
    spend in accepting input. If several mega bytes of data are being transfered max_input_time should be reasonably high. You can override the setting in the ini file for max_input_time by calling the set_time_limit() function in your scripts.

    Special Notes

    The apache webserver has a LimitRequestBody configuration directive that restricts the
    size of all POST data regardless of the web scripting language in use. Some RPM installations sets
    limit request body to 512Kb. You will need to change this to a larger value or remove the entry altogether.

    using a perl or java server side component. PHP happens to be our favourite web programming language as well but perl is just slightly ahead when it comes to file handling.

    Most installations of perl as an apache module can accept upto 32 megabytes out of the box. Compare this against the 2MB default for PHP.
    The downside is that perl coding takes just a bit more effort than PHP but it's worth it.


  • well the files are so large because we are dealing with wavs alot of the time and remember the song is being recorded. so it's not one file.  it's one for bass guitar 3-7 for drums 3 for back ups 3 for guitars and so on and so forth. so it can end up being alot of 4 min wavs.  I am definatly not a program savy guy so I'll have to read over this a few times and try to figure out my best option.

  • Ahhh.. now you're re talking more ;-)

    You still didn't have to upload ONE BIG file which is your session's mix.

    So if you're doing 32 trks => 32 X 4 min pieces => 32 files x 19 MB = 608 MB <== 1 file per track ;-) You still have the same as the Multi Tracker's component files.

    So you write a an Elgg page for each song / arrangement -- and embed media ( I think ) into the page or have link to yout WAV (Elgg) Files => 32 somewhat files ;-) Ideally you need a "threaded" FIle area so you can string the tracks as components of the "grouped" File.. but Elgg does not do that.. yet ;-(

    Having them in 1 HUGE FILE may be a little convenient but  does not serve any real audio / music purpose for you while living on the server -- because your server is *not the multi tracker / mixer / playback => server only knows "I've got a file to store... I've got a file to fetch... but I can't play that file..."

    ( BTW..  I am speaking from real experience,, used to run a digital + audio 16 tracker part-time ;-) Still have my (Steinberg) Sequencer, 3 guitars, bass, (Roland) sampler, 2 midi sound modules... and my 150+ originals.. LOL ;-)

    Check this out.. courtesy of my friend MalagaJack en ESspana

    ;-)

  • yeah I was hoping not to have to upload single files but to upload zipped packages.  I was entertaining what your saying (and still am).  the hard part is that the sight is for music guys not computer guys. so I was trying to make it easy.  I guess what I should do is make the site the way you are saying  and add as much documentation that I can and hope for a new feature in the future of uploading zipped packages.