Composer, dependency management and class autoloading - lessons learnt

In case anyone interested, here is how I currently feel about Composer:

1. It's a great tool for managing project-level dependencies, but will likely result in extra hassle on a plugin level.

2. Plugins should identify themselves using composer.json, so that they can be required by the project through packagist.

3. Composer may not always be the best solution for dependency management on a plugin level.

a) It seems that having the latest stable release of the vendor library is not always such a good idea. If used, dependencies should probably specify the release (sometimes running composer update results in unwanted updates that break otherwise working plugin features).

b) It creates an extra step. Almost always, you forget to run composer, run into WSOD, and have to see the logs to figure out that you forgot to run composer. More often than not, you can live without composer.

c) Likely dev dependencies are already included on a project level. Unlikely that you will be testing your plugins without the engine present in some form, so just recycle PHPUnit and co from there. No need to overload the plugin with an another instance of testing/dev tools.

4. If composer is used for depedency management within a plugin, vendor directory and composer.lock should probably be included in the repo (for convenience), and for the sake of being able to git clone without having to run composer.

5. Use elgg_register_classes() rather than autoloading with composer. Again, unlikely that your plugin can exist independently from Elgg. Especially when you have no dependencies, this helps avoid using composer for anything other than plugin identification.

6. Not sure composer installers do a good job at properly resolving root level and plugin level dependencies and avoiding duplication.

7. Using composer as a dev dependency tool in CI has a potential, but still requires a lot of work, especially when plugins depend on other plugins.

  • my 2 cents

    I like composer for plugins which have vendor files. This way i can reduce the number of files in my git project (point 4). If i configure the requirements correctly i don't run into all that many version problems (point 3a).

    A point from the composer website

    Should I commit the dependencies in my vendor directory?

    The general recommendation is no. The vendor directory (or wherever your dependencies are installed) should be added to .gitignore/svn:ignore/etc.

    https://getcomposer.org/doc/faqs/should-i-commit-the-dependencies-in-my-vendor-directory.md

    I made a travis build script that if i commit a tag it builds a zip-file with all the dependencies in it. So releasing is easy ;)

  • This doesn't solve the problem of multiple plugins with identical dependencies. You end up with a vendor library included in each plugin, and as a result you never know which one ends up in spl, you may encounter version mismatch.

    Until there is better support for composer in core, I am retiring composer in favor of custom class and vendor autoloading. Just keeping composer.json inside my plugins, so that they can be pulled in from root.

  • Ad. 2. I could help here a bit when I finally finish my neglected project of making community a standalone composer repo itself https://github.com/Srokap/community_satis. Apart from that it's much more preferable to encourage more devs to put their plugins on packagist.

    Ad. 3. Very much true. We still lack proper way of handling non-php stuff, ie. bower dependencies in plugins or similar. Any ideas on that part are very appreciated.

    Ad. 3.a That's what composer.lock file is meant for. Making https://github.com/Elgg/Elgg/issues/7721 happen should help here. If you're desperate you could always require specific versions in project root composer.json

    Ad. 3.b I see the problem. For a deployment process it should be a no-issue, but multi-devs team I see it could be annoying. I don't think it's much better way here if you want to avoid comitting everything what you build into repository.

    Ad. 3.c It actually should be a non-issue. In project context deps land in root /vendor dir so if two packages required it, you end up having single lib that satisfies both requirements. To make use from it in a plugin you should assume two possible places where your libs are, either in local /vendor inside the plugin dir or in root of the project. Ie. see WIP autoloader for Elgg to handle those cases https://github.com/mrclay/Elgg-leaf/blob/Srokap-elgg_application/autoloader.php

    4. That relates to the scenario of determining where /vendor actually lies

    5. Very much true. Composer autoloaders are static, don't want to have in there classes that come from disabled plugin. You could also rely on Elgg to register plugin classes as I don't see it going away anytime soon. If you need to autoload for plugin local tests, you can use https://getcomposer.org/doc/04-schema.md#autoload-dev to not pollute main autoloader.

    6. I don't understand the concern. Resolving deps works fine with elgg-plugin installer. If you manage to break it in any way, please send PR to https://github.com/Srokap/elgg_dummy1 and https://github.com/Srokap/elgg_dummy2 to reproduce that.

    7. Again don't understand the issue. It might have something to do with relying on having vendor dir within the plugin directory. If it were to be especially annoying when you do require on libs, maybe this could help a bit https://getcomposer.org/doc/04-schema.md#files Never used it though.

     

  • I will do more experimenting with the elgg-plugin installer when I have a chance.

    7. That relates to Travis. It seems like too much shell scripting to get basic CI up and running. Bootstrapping with core and dependency plugins (especially if you want to run simpletest as well) is a bit of a time waster. For now, I am just running tests locally.

  • Need to get Elgg more composer ready for 2.0.... Problem IMHO is that Elgg is installed at root instead of installed as a dependency. And also that we have not yet pushed composer as the one true way to install and manage dependencies. Your custom project stuff should go at root (site local modifications is another thing we don't currently have a good solution for).

    I.e. if you're working on a plugin, that plugin should be the project. There shouldn't be a difference between plugin level and project level.

    Elgg is hard to bootstrap for testing in ci because it relies on globals so much... It ought to be trivial to create an in-memory instance of Elgg db that you can play with without all the mysql hassle. So you can at least do basic integration testing... Sorry to hear this is affecting your usage of composer...

    I actually don't see much harm in registering classes from "disabled" plugins. If they're truly disabled then the classes should never get loaded so they can't affect execution of the project.

  • I was relying in the past on class registration order to override already registered class path with the one from plugin with higher priority. It served as entry point for internal framework bootstrapping to avoid having technical plugin just for that. Just loading everything would make things simpler for the core though.

  • I was relying in the past on class registration order to override already registered class path with the one from plugin with higher priority.

    Initially I thought of this as a feature, but I no longer feel that way... Classes, IMO, should be statically analyzable, and if you can override a class definition at runtime then you render static analysis significantly less helpful. Same with functions. There's no way to override it at runtime. A better approach for trading implementations is interfaces/polymorphism.

  • The wsod has been taken care of as we give a helpful error message if we detect composer hasn't been run yet.

  • The need to include two incompatible versions of the same library is an interesting one. The bower-asset composer plugin has a way to handle this but I haven't checked whether composer can do this for normal php packages.