Oh and we are not going to modify ELGG's core. Just modify it in terms of plug ins .

Any framework has its learning curve (whether Elgg, Zend, etc.) and there will always be times when you prefer to do something one way and the framework is designed to do it another. I'd argue that any really successful website based on a framework will eventually need to have chuncks rewritten to scale for high traffic. Facebook rewriting parts of PHP is the extreme example.

As an application-focused framework, Elgg is fairly flexible - not as flexible as a general purpose framework like Zend, but probably more so than other similar frameworks. I expect that the vast majority of websites built on Elgg won't have scalability issues given decent hardware. With a video application, your scalability issues are going to come from storing and serving the video before you hit any issues with Elgg.

Elgg's data model is very general purpose. This makes it really easy to write plugins to extend Elgg but there are those times when you need a little more flexibility - whether for performance reasons or whatnot. We have added some tables with plugins to get what we needed without breaking anything in the Elgg core, but there are limitations there.

The caching in Elgg needs work. It's not as flexible as the caching schemes provided by a lot of PHP frameworks where you can chose file-based or memory-based.

I think the best answer really depends on your team and the application. Hard to comment from the outside.

Cash, thank you for your input!