Repository navigation
Monolithic releases #3
Description
From #2:
A much bigger concern to me is the overall decline in activity and the decline in new active committers, and at a time when overall ecosystem growth and interest in node is soaring. I'd look at people down the list and ask what may have turned them off from the project or, more importantly, what might have encouraged them to stay and potentially even increase their contributions.
I contributed the vm module to Node 0.11.2 or something like that. My purpose was so that consumers of jsdom could stop relying on a compiled add-on. I would have been interested in contributing more to core, but quickly lost interest when it became clear that Node 0.12 was never going to be released, and so none of the contributions I made would ever be usable by people I work with (e.g. jsdom consumers, or clients of the consultancy where I worked, or fellow open-source developers). That's what turned me off from contributing.
From this perspective the correct approach IMO is to abolish the even/odd tick-tock and just release things that are ready when they are ready. Use feature-flags to e.g. keep vm in node@beta until the bugs have been shaken out, but once it's un-bugged, move it into node immediately, instead of making its progress depend on a bunch of other features in a big "0.12" release. Do node@beta and node-stable releases on a time-based schedule (e.g. every 6 weeks), and whatever's ready after that time period is what gets shipped.
Honestly this whole stable/unstable release cycle feels like such a standards-bodies-esque mode of failure, e.g. waiting to release ES6 arrow functions until all the details of ES6 modules are finished, just because someone decided to bundle them all under the name "ES6." Browser vendors have realized this is not a good model of working, and started shipping features independent of spec version; even TC39 is moving to a feature-based model. I wish Node would too, instead of these monolithic bundled releases.
Activity
stefanpenner commentedon Oct 10, 2014
I think you nailed it, good momentum is key to healthy contributions (from both new and existing contributors).
mikeal commentedon Oct 10, 2014
I actually agree with both of your points but I think they should be separated.
Both are bad, but I don't think the slow rate of releases is directly related to the even/odd numbering scheme. At the same time, the even/odd number scheme is super confusing because the entire Node ecosystem uses semver.
Fishrock123 commentedon Oct 10, 2014
Well, sorta-semver anyways. I'll leave this here: jshttp/style-guide#4
tl;dr: Semver means people should bump majors more, but people don't like upgrading majors.
(As a result of this and many other things express doesn't really use strict semver.)
mikeal commentedon Oct 10, 2014
Well, we should define what "using semver" means. I've probably never used it in the strictest sense, but he linear progression of version numbers I have stuck with, as has everyone else, because it gets enforced by npm.
This concurrent version number thing is very confusing for new people. We copied it from Linux before anyone even read the semver spec and npm existed. It's just straight up legacy and should be put out to pasture.
Fishrock123 commentedon Oct 10, 2014
Not the best thing, but yeah, to a degree.
I actually like the idea behind it. However, I am inclined to agree.
Though Node should be 1.0'd before we do this, otherwise stuff like 0.13 is gona be super confusing haha.
mikeal commentedon Oct 10, 2014
Absolutely, nothing would be more confusing than changing the versioning before declaring a 1.0. However, 1.0 should have been "declared" a long time ago :)
FotoVerite commentedon Oct 10, 2014
And gifs no longer works. So totally agree.
Qard commentedon Oct 10, 2014
It'd be nice to see Chromium/Firefox-style nightly/dev/prod release channels.
mvolkmann commentedon Oct 10, 2014
I SO want to use arrow functions in Node now! Don't make us wait for other ES6 features to be finalized.
domenic commentedon Oct 10, 2014
@mikeal I don't think they're as unrelated as that. The even/odd scheme---or really, any scheme that has an unstable branch where active development happens, leading up to a new release---is actively counterproductive to attempts to merge features in as soon as they are ready. The idea that 0.10 will get no new features, and thus we need to wait until 0.12 to get a new vm module---while the entire time a finished and polished vm module in the 0.11 branch is being held up by unfinished AsyncListener stuff---is the problem. If 0.10 was allowed to get new features, instead of being the stable no-new-features release train, I'd be much happier.
domenic commentedon Oct 10, 2014
@mvolkmann that has nothing to do with the current thread (please re-read my post) and doesn't even make sense anyway since V8 doesn't have arrow functions.
aheckmann commentedon Oct 10, 2014
The wait for 0.12 has caused me to think the project is running out of gas and losing focus. I disagree that adding new features in a minor release (0.10) is the right move because I like knowing new features / API changes are not going to impact me when I update a point release.
I too would like to see a schedule put in place to ship smaller, ready features more often.
mvolkmann commentedon Oct 10, 2014
@domenic, you said "Honestly this whole stable/unstable release cycle feels like such a standards-bodies-esque mode of failure, e.g. waiting to release ES6 arrow functions until all the details of ES6 modules are finished, just because someone decided to bundle them all under the name "ES6."
That's where I got that idea that the Node team had decided not to enable use of arrow functions until other ES6 features were finalized.
stefanpenner commentedon Oct 10, 2014
Assuming these features are purely additive they should not disrupt existing code, if they do they should be considering breaking and clearly not included in a point release (and maybe not included at all)
Qard commentedon Oct 10, 2014
Even if the feature is stable, it should stay marked as experimental until
the next minor release. It's easy for users to lose track of what versions
support a feature they want to use, if it could have come in a patch
release. Following semver properly should make it easier to track.
4 remaining items
Fishrock123 commentedon Oct 11, 2014
Ok so I have a serious question.
If we do 1.x.x / semver with a quicker release cycle, which versions are we going to support for back-porting fixes?
bnoordhuis commentedon Oct 12, 2014
@Fishrock123 asks a great question.
Let me start out with the related issue of upgrading V8: a v1.0.0 release is a promise of stability but it's not feasible (nor desirable) to freeze V8 for all eternity. People want features, distros want security fixes and maintainers want support from upstream. But upgrading V8 inevitably breaks tons of add-ons.
As a conversation starter, I'm going to suggest allowing V8 upgrades (and libuv, openssl, etc.) with every minor (not major) release. TBD how often to put out minor releases.
I've become a fan of time-based releases; it's not perfect but it gives developers a deadline, users a date to look forward to and it avoids the "release when it's ready" quagmire that joyent/node is stuck in. How does a minor release every four months or thrice a year sound?
That brings me back to @Fishrock123's question, support for older releases. I suggest bug fix support for $minor-1 and security fix support for $minor-2. That's eight months of full support and one year of security fixes.
Fishrock123 commentedon Oct 12, 2014
If this actually becomes reality.. :D
My observations so far is that "release when it's ready" just doesn't work in software, no matter what the software is.
Ok maybe if you're programming the mars rover. Then it might be applicable. :P
So what about major bumps? Inevitably there will be people too quagmired in complex legacy code to update for all the breaking changes.. (See 0.8..)
@bnoordhuis What's the sorta time-plan-idea for majors?
jonathanong commentedon Oct 13, 2014
a time-based release sounds awesome! another approach is just to release a new version whenever v8 updates to a new "stable" minor version. i.e. it looks like v8 3.29.x took about 1.5 months.
i'd rather have a v1.0.0 quickly and do major releases whenever there's a possibility of a breaking change. i'd also rather core not be afraid of adding/removing/changing APIs, but they should make it easy on users via a utility like: https://github.com/dougwilson/nodejs-depd
Fishrock123 commentedon Oct 13, 2014
Yes, using something like
depdwould be excellent. +1domenic commentedon Oct 13, 2014
@bnoordhuis glad you're a fan of time-based updates :). I agree for all the reasons you state.
IMO four months is way too long to wait for new features or for a V8 upgrade. Six weeks would be much better; as @jonathanong says, that is how often V8 releases. Four months would mean in some cases you wait a third of a year for something you've been using in Chrome for that long.
This is purely from a non-native module author and consumer perspective, though; I understand the way in which V8 upgrades cause breakage could be a cause for slowing things down. But I want to present a counterbalance to the "take things slow" crowd :). So with that in mind:
In a rapidly-updating world like this, I'd guess that native module authors would become more active in un-breaking their code, but there would still be some lag from native module authors and from those who consume them. So you'd probably need to have infrastructure in place to backport fixes efficiently. I might do security fixes only though, to reduce workload and indirectly help encourage the ecosystem to upgrade?
An abstraction layer like nan or similar bundled with core could help reduce the friction in order to make upgrading more seamless, and could even cause some "breaking" releases to convert into non-breaking if the differences are able to be abstracted behind a nan release. That is, you could go by a few six-week cycles before reaching a point where the changes are too big for nan to encompass.
aredridel commentedon Oct 13, 2014
bundling nan or similar would help a ton, if we can provide an API surface that we bless as 'stable'. A good part of following semver is declaring what things are not supported interface. If v8's specifics are "out of zone", then it's easier to follow semver -- but reduces what one can do with the system and stay stable. Trade-offs.
hunterloftis commentedon Oct 16, 2014
I agree with @domenic that the versioning scheme is tightly coupled to forward momentum. Even/odd turns the unstable branch into a second-class citizen, discouraging progress:
https://github.com/mongodb/js-bson/issues/64
Additionally, even/odd discourages selfish patches. Many of us write selfish PRs - we want to use a feature now or fix a bug now, for ourselves or our customers. That motivation disappears with even/odd versioning.
Finally, batching features together creates artificial delays. In some cases, I'm encouraging Heroku users to deploy Node 0.11 because it has features that can really benefit them today - and waiting for 0.12 could mean waiting indefinitely.
I respectfully disagree with @Qard that nightly/dev/prod release channels are a solution. I actually see a lot of confusion about Node versioning already, and I'd like to simplify and streamline it into a linear (semver) path, if possible.
Regarding release schedule, I think @bnoordhuis has the right idea with time-based releases, but I'd love to see updates more than three times a year. However, he would have a much better idea than I do about what's feasible in terms of core development. If node moves to a time-based, small-features-fast, semver-organized release schedule, I would be very interested in contributing to core myself.
Qard commentedon Oct 16, 2014
You can do nightly/dev/prod with semver. Just use beta and alpha suffixes.
I'm not super adamant that is the best way. I just suggested it because it
seems to work for Chrome and Firefox.
bnoordhuis commentedon Oct 16, 2014
It's mostly a function of man power. We discussed a 6 week release schedule in yesterday's TC meeting but that raises the question of how long older releases can be supported.
Someone (I think it was @piscisaureus) suggested doing LTS releases. I think that's a good idea but the details of how often and how long to provide support are still TBD. Input welcome.
Qard commentedon Oct 16, 2014
Perhaps you could set whatever the last minor version was as LTS whenever you bump major.
For example: 1.5.x is out and V8 changes everything again. You lock 1.5.x as the current LTS and bump to 1.6.x before introducing the breaking V8 changes.
If necessary, you could have a two-stage LTS system, where there's an actively supported LTS and a deprecation warning LTS. Both would receive security patches and such. If using the deprecation warning version, it's advisable to switch to active because the LTS window could shift off of it at any time.
bjouhier commentedon Oct 16, 2014
@domenic Agree. It takes too long to get from 0.11 to 0.12 (still hoping we'll get there). It's not really a problem for us but it sends a bad signal.
Would be better to have a new stable branch at a regular rate (every 6 months would be good for us but faster would not hurt).
It is important to get updates on the stable branches (only bug fixes should be allowed). It is also important to encourage people to upgrade to the latest stable branch. This can be achieved with a policy like "bug fixes will be applied only to the last N stable branches and security fixes only to the last M stable branches" (M > N).
In our experience upgrades are rather smooth. The main pain point is V8 API changes.
rlidwka commentedon Oct 20, 2014
Even-odd system is great. Only issue is that we rarely have releases nowadays, but it has nothing to do with the versioning system.
Another idea would be to bind node versioning with v8 versioning somehow. It will certainly make sense now when everybody waits for the es6 features.
darrenderidder commentedon Nov 16, 2014
I generally like semver, but whether it goes with even/odd or stable / dev branches, an LTS version would be really important for the big enterprise-type users. A one-year rolling window for security patches is probably too short for the large international carriers we're deploying into, so I'd like to see security fixes back-ported to an LTS version if possible...
mpareja commentedon Nov 16, 2014
I can't help but think that a lot of these flow issues would be minimized by maintaining a visible backlog and appropriately breaking up the work into small independently deliverable chunks of value. Maybe I just don't have visibility into the right conversations, but I get the sense that work happens and is tracked in terms of monolithic features. Thoughts?