Repository navigation
Release cadence due to COVID-19 Pandemic #553
Description
Hey All,
Was curious if we should consider adopting a different release policy for the foreseeable future in light of coronavirus and COVID-19. For example chrome is currently delaying upcoming releases due to the current situation.
One thing to consider is that we want to limit the amount of work that downstream consumers need to do to keep Node.js up to date and their services running
Some questions we need to ask:
- Should we delay 14.x?
- Should we put a moratorium on new features?
- Should we temporarily make all LTS "maintenance only"
Stability in our release lines will likely help folks downstream not have additional fire drills during a trying time, alternatively creating too much of a backlog could create other hardships down the road. So to me it seems like we could do 3 different things (more options welcome of course).
- Change nothing, keep currently release schedule and candence
- Continue to release Current (e.g. 14) at planned schedule but limit LTS releases to maintenance only
- Feature moratorium across all release lines, security fixes + major regressions only.
Thoughts?
Activity
codebytere commentedon Mar 19, 2020
I think it might be prudent to consider that our productivity may generally be lower as we collectively deal with all the other externalities created by this pandemic, and so we may want to optimize for what we consider to be best for the ecosystem. Chromium and Electron are both pressing pause, and I would imagine that V8 may be as well.
I think I personally would be in favor of reducing feature-motivated changes to existing codepaths as much as possible in agreement with this:
as I think consumers will have a harder time maintaining recency and less bandwidth to address issues. I think i personally would be in favor of 3 but also happy to pursue 2, though there are most likely things i may not be considering or aware of and would want to know y'alls thoughts as well!
targos commentedon Mar 20, 2020
I think 2 is reasonable if we apply it after all the releases that happen next week.
[-]Release cadence in lieu of COVID-19 Pandemic[/-][+]Release cadence due to COVID-19 Pandemic[/+]BethGriggs commentedon Mar 20, 2020
+1 as a starting point.
We could also consider at increasing the time between current releases (every 3-4 weeks, rather than 2), if we see that as appropriate.
BridgeAR commentedon Mar 20, 2020
+1 from me as well.
Downstream mostly uses the LTS releases (as can be seen looking at the download numbers) and therefore limiting those seems fine to me. I would keep on releasing current as frequent as possible though to get people to test new things as soon as possible. Earlier and therefore more testing will likely prevent regressions to "leak" into the TLS releases.
indutny commentedon Mar 20, 2020
+1 from me for 2. We should minimize the chances of breaking others code.
sam-github commentedon Mar 20, 2020
I'm not sure who we are trying to help with this. Certainly not people waiting for features to be released so they can use them.
When we release new features, it doesn't cause any extra work, does it? People can adopt the new releases, or not, can't they, as they have time and inclination?
If as a project we get less contributions because people have other things on their minds, that's OK, Node.js can continue to release when we want (and when volunteers step up to do the release), and the releases will continue to contain what's ready. If that's less things ready, that's OK.
davidmarkclements commentedon Mar 20, 2020
@sam-github breaking changes can cause issues, moving versions into maintenance or end of life tends to trigger the need to update and beyond that new features can absolutely cause more work. Think about platforms that provide sandboxing services etc.
MylesBorins commentedon Mar 20, 2020
@sam-github in just the last month I've had to do multiple "emergency" follow up 13.x releases as changes to streams broke people down stream. The last 12.x release also required a follow up as it broke a bunch of things (we had to do a post-mortem).
This caused extra work for our team, although that is expected, but also lots of work for folks downstream who needed to upgrade, test, and report breakages.
If a security release drops after a couple of these releases that breaks things it does become much harder for folks to adopt them. My concern is that a faster release cadence on our end could end up with folks downstream sticking to older versions to avoid extra labor while they are focused on other things.
bnb commentedon Mar 20, 2020
As someone who works to help enable product teams to more aggressively ship Node.js releases as parts of their various platforms, I would like to share a +1 to slowing down however the release team feels is most appropriate for y'all.
This is incorrect. People who are implementing Node.js as a part of a platform often end up with a non-trivial amount of work when features are shipped that is not simply opt-in - regardless of whether they're large features or not.
Whether that platform is a cloud or developer tooling, every time Node.js ships a feature there is churn for engineers who have to prepare both for potential unexpected stability changes as @MylesBorins mentioned and for their consumers to use any kind of new features that we ship.
sam-github commentedon Mar 20, 2020
Is the code flow into projects like Chromium and Electron dominated by single companies? If so, those companies can choose to throttle the rate of contribution, as well as release rate, so they aren't creating bottlenecks.
Node.js contribution is quite distributed. Its hard to get a read on rate of incoming changes, but I haven't noticed a slow down yet. As with everything related to COVID19, the situation is unprecedented and unpredictable.
Releasing less often will likely mean that when releases do come out, they will be much larger. That itself has some risks, but if we are hearing downstream consumers say larger releases, less often, are more easily consumable, I've no objection.
mhdawson commentedon Mar 20, 2020
I'd add another option for consideration:
Continue to release Current (e.g. 14) at planned schedule but limit LTS releases to the model for maintenance + specifically requested/important changes only.
The main different being that if there are features that have and advocate and for which there is a reason to include them we still can versus maintenance only which would make that more difficult even in cases where it makes sense.
It still has the problem of future
largerreleases as @sam-github mentioned or the possibility that 12.x will never get back out of maintenance making the move to 14.x potentially a bigger jump/challenge from 14.x but is a bit more flexible than full maintenance mode.joyeecheung commentedon Mar 20, 2020
+1 to option 2 from me. On features, I think if the conclusion is the productivity is going to be seriously affected in, say, the next 2 weeks, we can just hold off landing any big features during that time frame (i.e. the regular feature freeze used by Chromium, which is happening as an exception due to the COVID19 situation), and revisit after that. If we indeed decide to release as few features as possible for a short period of time, simply stopping the release of them but still keeping them landing in master would be unhelpful because then we could end up with backport headaches. We don't need to settle down on one specific plan over the course since this is probably going to take long.
bcoe commentedon Mar 20, 2020
I think it's important to be cognizant of the fact that engineering teams are going to be significantly slowed down right now, and that new release lines can create a significant amount of engineering work: lambda, Azure Functions, and Google Cloud Functions will want to add v14 support, CI/CD systems will need to add it to their matrix.
I think it might at least be worth preparing ourselves for the v14 schedule to slip, and we should evaluate based on how things are looking over the next few weeks -- being mindful that it's not just the work for the Node.js project, but upstream folks as well that we impact (@bnb echos this point of view).
I agree with @joyeecheung, not that folks shouldn't open up PRs (some folks are finding coding a good escape right now), but I think a moratorium on releasing features to master right now would be good; concentrating on critical bug fixes/security patches.
MylesBorins commentedon Mar 20, 2020
So, fwiw, my personal thinking at the moment is that we could leave 14.x and the current release line at large unchanged, and just slow down LTS. As mentioned by Sam coding can be a good outlet for folks right now, and we don't want to halt all innovation or stop the project from moving forward. What we do need to ensure is that our LTS consumers are not going to be broken or have any major fire drills. If we stick with the planned releases for Tuesday the biggest thing that we could do is delay the April 12.x Semver-Minor release by at least a month. Another thing we could do is push off the start of 14.x LTS, but we have some time to figure that out
2 remaining items
richardlau commentedon Mar 24, 2020
V8 8.1 has been demoted to beta: https://groups.google.com/forum/#!msg/v8-dev/QCzlW_toPTY/Xk5-pggnBgAJ
What does that mean for Node.js 14 if we were to stick to the current schedule? Master currently has V8 8.1 and we've never landed V8 8.0.
MylesBorins commentedon Mar 24, 2020
I've been in touch with various individuals on the V8 team and have received a thumbs up on us sticking with 8.1
mhdawson commentedon Mar 25, 2020
@MylesBorins can you clarify what you mean by that. Do you mean that it will be out of beta by the time we ship 14.x or that you are saying that its ok to use a beta in 14.x based on discussion with the V8 team?
MylesBorins commentedon Mar 25, 2020
mhdawson commentedon Mar 25, 2020
Going to mark for TSC agenda so that we have discussion on whether our project is comfortable shipping with a beta.
mhdawson commentedon Mar 26, 2020
To clarify, the decision on using a beta is in the scope of the Release team, but I think it's an important enough change from our regular process that I think it's important the TSC has an FYI which is why I've added to the agenda.
jasnell commentedon Apr 1, 2020
Sorry for being late on the discussion, I'm definitely +1 on maintaining the regular schedule for current releases but restricting LTS updates to maintenance through the remainder of 2020.
MylesBorins commentedon Apr 3, 2020
So here is a summary of the consensus we had in Thursday's Release WG call. We should likely get this out in a blog post, I've been meaning to write it and will try to do so tomorrow.
codebytere commentedon Apr 3, 2020
@MylesBorins i can also try to take the post if that'd be easier!
MylesBorins commentedon Apr 3, 2020
BethGriggs commentedon Apr 14, 2020
Blog summarising the changes that we made to the schedule - https://nodejs.org/en/blog/announcements/adjusted-release-schedule-covid/
We will continue to review the schedule.