Branch support after 2.x

So... I've noticed with 1.x we now have several parallel branches to deal with some a new feature branch is created every 6 months. This is a fair bit of overhead for our small team.

How would folks feel about dropping branches for minor releases, and only supporting major release branches? I.e. we release every two weeks as now, but these might include features, too? That means we don't add a new branch to support every 3 months. It's just one branch all year long (or, until the next major version bump).

  • We stick to semver of course, so the versions may go straight from 2.0 to 2.1 without any bugfix releases in between, though of course bugfixes would be included in the release.

  • Site owners would lose the opportunity to receive fixes with relatively low chance of introducing other regressions. That said, they're only getting that service at most for 3 months now, assuming they install/upgrade to 1.x.0 immediately upon release.

    Security fixes would have to trigger 3x the releases they currently do.

    In exchange:
    - Feature additions wouldn't need to sit waiting for months to come out. Huge benefit for everybody as far as I can tell.
    - We don't have to reject fixes that have a bit of refactoring in them.
    - We don't have to the go back and re-implement the simple fix in a better way.

    I don't see this adding work for plugin devs as we promise APIs for the same amount of time. Though it'd be annoying if we deprecated an API potentially anytime.

    We should definitely chew on this, but I like it. We should brainstorm other ways to reduce our project management overhead.

  • Wouldn't security releases mean we need to keep all those branches around anyway? Otherwise we have to checkout and cherry-pick onto tags.

  • Or we could make people upgrade all the way up to the latest 2.x to get security patches. I could see that going one of two ways:

    • Admins are so concerned about stability and so unwilling to upgrade across minor releases that they forgo the security patches for a while (very bad).
    • Admins are so compelled to get the security fixes that they also take the time to upgrade to the latest and greatest and get all the feature benefits, too (very good).

    Theoretically we'd be only committing changes that are stable, and since each change gets out there very quickly it is more likely to be tested in production for a while when the security patch does come out, so we'll have a good chance to fix any upgrade issues.

    Anything that seems unstable or could cause problems we can always push off to the master branch, or ask for more tests before pulling in, etc. Basically if you think it requires a bunch of manual testing before pulling in, it should go on the master branch because that's the only branch we'll do pre-releases from. Pre-release == crowdsourced manual testing

  • Is it all about having an even faster, faster, faster release schedule? At least that's what I read between the lines...

    Forcing site admins to update to the latest available release to get security fixes? And what about 3rd party plugins then? Is it realistic to expect that you can update to the latest Elgg version immediately all the time or isn't it more realistic that you might have to wait possibly for a few months at least sometimes until the 3rd party plugins you are using have catched up and you are finally able to update Elgg core?

    New features with a certain risk of regressions (even if the risk is small) and possibly sometimes with deprecation of APIs added on an unpredicable schedule? Again: what about 3rd party plugins and specifically what about 3rd party plugin devs? With the current 3 month release schedule of feature branches it's already seriously too tight for my liking (and some other devs seem to feel the same...). Even with semver you can never be 100% sure that a regression in Elgg core does not results in an incompatibility within a 3rd party plugin. Even when testing only briefly (which is not the best way in the first place) the testing of 3rd party plugins on new Elgg versions takes a considerable amount of time (which is taken from coding time). And if you don't have to test only with new feature branches on a specific number of occassions per year but basically with every new Elgg release it gets simply impossible to handle anymore.

    It's not only the Elgg core dev team that is small. The number of 3rd party plugin devs is very small, too! And spending a lot of time with catching up with Elgg core again and again instead of being able to code on new features within plugins or to create even new plugins for a change is demotivating and might result in the number of 3rd party plugin devs getting even smaller and smaller sooner or later...

    Is it really too long to wait a few months (not years!) until new features get released within a new feature release. Personally, I wouldn't be disappointed at all to get a new feature release only every 6 months instead of every 3 months. A 6 month release schedule would already decrease the amount of work necessary to maintain different branches in parallel simply because there are less branches to be supported in parallel.

    Regarding the support of older branches it seems to me that it's mainly a matter of security fixes that is to be dealt with because bugfixes are no longer provided for the older branches once the next feature release gets released anyway. And it seems that security issues happen very, very rarely fortunately.

    Bi-weekly bugfix releases. I've said it before and I say it again: the bi-weekly releases schedule is too tight. As it is, mostly too tight to even make a new release every 2 weeks. That's not meant as critic but just as a realistic observation of the facts: since the bi-weekly release schedule has been started it has happened only twice or 3 times that 2 releases were made within 2 weeks. In all other cases there were 3 or even four weeks betweek the releases. So, why continue to stick on the bi-weekly release schedule? Why not officially make a bugfix release only once per month instead? This seems still short enough and would reduce the workload for making new releases by 50%.

  • I also think a monthly bugfix release would be fine. Dropping bugfix releases altogether I'm not convinced would be a huge timesaver.

    Retiring all the branches not maintained for security would clean up the list quite a bit.

    I also feel a dropoff in the number of devs working in Elgg in general and I think this is due to the product trying to serve two communities (pros using it as a framework, and website owners who just want a social site) and really only serving the former.

    The Showcase shows that capable teams can build great stuff with Elgg, but the out-of-box UX is just not there. I think we should consider theme extension plugins that are bundled but only enabled during the initial installation, so they wouldn't be promised as BC (or we just release them separately). aalborg_2, aalborg_3, etc. We need to innovate between major releases.

    @iionly Can you point me to complaints from other devs about the increase in minor releases? I think you may have a higher standard of QA than most plugin devs. If you want to provide end users with top notch support for free, that's honorable, but it sounds personally exhausting.

  • On a brighter note I think 2.0 is going to be the easiest version to develop front end for. A plugin to replace all the images across Elgg would be trivial now.

  • Hey, iionly! Thank you for your perspective. You are a big help at keeping us considerate of the plugin developer community. Clearly folks out there feel a weight that the core team is not appreciating (lots of releases == stressful).

    My main takeaway from your comments is that too much time is being wasted on manual testing. However, the solution to this in my mind is not "require testing less often" but rather "make testing cheaper" by automating it.

    Let's assume we could 100% verify that a commit doesn't break plugins. Is there any other reason why wouldn't we want to release that to the world ASAP?

  • @Steve I like your idea of a separate theme that we can use to innovate on UX between major Elgg releases.

     

  • @Steve concerns were told me in private conversation, so I can't refer to it.

    I don't know if I have any higher standards of QA than others but I just don't feel good about stating plugin compatibility with a new Elgg version if I haven't at least tested it to a certain extend. With the 1.X releases since 1.10 I'm beginning to trust that compatibility is preserved. But it would be a completely different matter if there would be kind of a "rolling releases" model adopted for 2.X with not only bugfixes but also new features getting added at any time.

    @Evan somehow I expected the argument of automated vs. manual tests when I made my previous posting. But I don't see manual tests getting obsolete in the near future. Automated tests might lesses the effort needed for testing but I don't see any chance to omit manual tests completely to ensure there are no unexpected interferences between Elgg core and 3rd party plugins (as automated test will most likely always be limited to test rather expected interferences only).

    Let's assume we could 100% verify that...

    I don't think there will be 100% safety ever to be achieved. And it just makes it so more difficult for plugin devs (and I think for users, too) if new features get added on a completely unpredicatable schedule.

Feedback and Planning

Feedback and Planning

Discussions about the past, present, and future of Elgg and this community site.