How regularly would you like to see releases happen?

Summary: The core team has voted and approved a 2-week patch release cycle and a 3-month feature release cycle.

I've been plugging away on https://github.com/Elgg/Elgg/issues/6783 to help make it easier for us to release more often. We're at a place, I think, where we could release much more regularly so that you're waiting only weeks for patches and months for features, rather than months and years respectively.

What is a good frequency for major (breaking), minor (feature), and patch (bugfix) releases?

Update (2014-05-10): Thanks, everyone, for chiming in. Lots of good discussion here and that is really helpful to get all these perspectives. Here’s my summary of the discussion so far:

Unfortunately there's no clear consensus on how often to release from the various branches. :( Bummer. That means whatever the core team decides will end up disappointing some people, which is less than ideal.

Motivations for frequent, regular releases:

  • Avoid code going unused for months/years
  • Encourages core contributions if people know their patches will be available sooner
  • Project looks more active, which improves trust and helps drive usage/contribution
  • Our current release process is a horribly negative cycle:
    • Releasing takes too much time which results in putting it off
    • More changes are included while we’re waiting for the release
    • Since more changes are in, we need to test even more thoroughly
    • We put it off even longer because people don’t have time to test + release
    • Releases get blocked on features that people don’t want to slip until the next release
    • They end up waiting anyways
  • Incentivizes automated testing, reducing long-term overhead
  • No frequency works for everybody. Better let admins have the choice.
  • Allows admins/devs to decide how often to update, rather than artificially limiting them.
  • Shipping is a feature.
  • Makes it much easier to determine which change caused a given problem.

Concerns about frequent, regular releases:

  • “non-breaking changes” really means “not-intentionally-breaking changes”. Stuff happens
  • The current test suite for Elgg doesn’t guarantee enough stability.
  • Evaluating and testing changes for a weekly release would be too burdensome.
  • You can already pull from git if you want bleeding edge update
  • Looks better to clients if you are up-to-date with the latest release, which won’t be feasible.
  • Frequent releases look bad. Suggests there are lots of bugs
  • Faster releases will result in unstable code
  • Current testing practices are not sufficient or non-existent
  • Some people feel very compelled to keep up-to-date with new releases
  • Discourages developers who feel like they can’t “keep up”

Obviously we need input from the core team as well because this is not a totally-automated process quite yet, but we definitely want to get there as much as possible so we're not holding back features or patches for trivial reasons.

We're moving to semver after 1.9 is released, so there will be 0 breaking changes to the public API until 2.0. You should never need to update your plugins until the next major release. We still reserve the right to break the private API at will, so be careful about undocumented "features" that you're taking advantage of. If you're doing this, submit a bug report to get it documented/supported as an official API.

Also: we are going to strive to keep release branches very stable so that nothing blocks releasing like we've experienced these last 5 years! This stability may mean your PRs take longer to get in, because we will not accept half-done features even with a promise of a follow-up PR to wrap things up. They need to be done before they get in. But hopefully that's ok because you can be confident that once they are merged in, people will actually be able to use them very soon afterwards.

My strawman suggestion:

  • patch releases once per week
  • feature releases once per quarter (1.10 in Sept 2014, 1.11 in Dec 2014)
  • breaking/unstable/risky updates no more than once per year (humble 2.0 release sometime in 2015).

Discuss.

  • I have no WorPress knowledge, but apparently with 60 million websites they must have thought this through. Webservers having write access to document root is as fas I know very common.

    But back on topic. I think regular updates should not be a goal on itself. Auto updates could be.

    - Security and major bug fixes should come out immediately (being as fast as possible). These fixes should preferably packaged as patches and should not require a full system upgrade.

    - Other (minor) bug fixes and functional changes have less priority and no fixed time limit should be applied here.

    - Major upgrades can and have followed their own schedule so far and from a user perspective it is always nice to have this as soon as possible but it is limited by the number of developers involved and the time they can spent on working on this. That is the nature of this project.

    Stability is then covered, since security and bugs are a threat to stability and therefore should be applied immediately, also by conservative commercial customers.

  • I really agree with Gerard for the most.

    • Security and majorbug 'patches' as soon as they're patched.
    • Functional/minor changes every 2-4 weeks, automaticly.
    • Major upgrades following the coding schedule.

     

  • I voted on the release frequencies, but these are weak preferences. I'm not all that compelled by regular releases, but am highly interested in making the release process quick and painless. Core team involvement is so variable that I could see releases going out insufficiently tested if they're on a tight schedule.

    To bring up testing again, although we're making great strides and now trying to consider testability into our feature development, the complete lack of automated end-to-end testing (browser walking through each feature, submitting forms, etc.) puts our real coverage very low. You would think it would be hard to break the group edit form without anyone noticing, but I've done it (in a bugfix release!)

    Anyway, my suggestions for improving the situation:

    1. Nudge savvy developers to regularly pull from the patch branch instead of waiting for tagged releases, and make it clear that "you're getting fixes right away for the price of occasionally helping us find regressions before they're released."
    2. Start considering testing with something like Selenium + PhantomJS, focusing on the major features.
  • yup 4-week cycle or 3-week is best i think :)

     

  • Thanks, everyone, for your input. The core team has voted and approved a 2-week patch release cycle and a 3-month feature release cycle. I look forward to seeing smaller, more stable, more frequent releases coming out of this project. Should be a fun time!

This discussion is closed.

This discussion is closed and is not accepting new comments.

Feedback and Planning

Feedback and Planning

Discussions about the past, present, and future of Elgg and this community site.