Deprecation Policy

[mikeal]

Recently we've run in to issues related to deprecation and have realized we don't actually have a formal deprecation policy and even the one we are culturally enforcing now doesn't handle the breadth of use cases we have.

We should discuss and draft a deprecation policy and pull together various use cases as hypotheticals for it. Some recent issues that come to mind are:

• V8 4.4 to remove indexed properties via external data V8 4.4 to remove indexed properties via external data #1451
• NODE_PATH deprecation NODE_PATH deprecation (TC voted negative) #1627

[Fishrock123]

Also: util.is* (which I still haven't done a doc deprecation for..), sys, etc.

[rvagg]

I'd like to see us make 'sys' go away as a first step to making deprecation a real thing, it's been warned about for a long time now

The V8 4.4 issue is also interesting; for those not following - it's going to force us to completely remove 'smalloc' so we are put in a position of taking proper deprecation action.

[mikeal]

part of the sys deprecation strategy should be to identify modules still relying on it and, if necessary, send them PRs.

[chrisdickinson]

As part of this I'd like to try actually removing sys for v4.0.0 while doing a pre- and post- mortem of the process so we know what we're dealing with.

[Fishrock123]

Deprecation notices may be easily maintained, but it's not really a solution. +1 to @chrisdickinson

Maybe we should do a pre-mortem first so we can gauge where this is being used the most and already send PRs / info beforehand?

Also, regarding sys, I think I remember the bigger issue being removing the things it exposes?

[chrisdickinson]

Yep – I'll write up a pre-mortem, to include:

• How to identify the modules that need fixing.
• How to estimate the effectiveness of mitigation.
• What the fixing PR process should look like.
• Go / No-go states for removal, plus a deadline.

[jasnell]

@mikeal ... we should work on formalizing the deprecation policy in the dev-policy document further. Some of the processes are documented there but it's been an open issue. Perhaps it would be best to keep the high level discussion of the process there so that we can focus on the specific technical items here?

[chrisdickinson]

deprecating sys – premortem and plan of action

step 0: set goals

Our goal is to pick a minimum percentage of weekly downloads that should work
before we're willing to remove sys. Obviously ideally this would be 100%,
but our time on Earth is finite and divided between other activities that don't
involve GitHub, text editors, or the internet.

As an hypothetical example:

• Say removing sys breaks three modules.
  • Package FOO has 10000 weekly downloads.
  • Package BAR has 1000 weekly downloads.
  • Package SHAZBOT has 10 weekly download.
• Removing sys breaks 11010 weekly downloads.

My preference is to set this percentage between 80 and 90 percent of downloads.
We will call this the target mitigation number.

step 1: identify usage

Deprecating an entire module is somewhat easier problem than deprecating
an API that a module exposes. That said, I'll attempt to use the same tools
to find any usage of sys in the npm ecosystem:

1. Run estoc against every version of every package in
   the npm ecosystem. This requires a fullfatdb clone of the registry.
2. Grep estoc's output for "sys" usage.
3. Sort usage by download count.

step 2: mitigation effectiveness

The question to be answered at this stage in deprecation is "how much effort will
it take to reduce breakage below a certain threshold?" We must take into account that
fixing the broken package itself does not guarantee that the number of downloads fixed
will go up. We're more concerned the the dependents of broken modules.

To estimate the amount of effort it will take to reach our target mitigation number,
we have to understand the network of dependencies around our target modules.

What we want to know is the minimum number of packages that will have to release a new patch
version before we reach the target mitigation number.

Given:

B           = the initial set of broken packages requiring a fix
Deps(p)     = the set of dependent edges on package@version "p", a pair of (package, range)
Fixes(r)    = whether a range r will pick up a patch version release – 0 if no, 1 if yes

We run the following pseudocode walk through the graph:

E = <empty set>
A = <empty set>
for package in B:
  A.add(B)

for package in A:
  for (dependent, range) in Deps(package):
    if Fixes(range) == 0:
      A.add(dependent)
    else:
      E.add(dependent)

The set, A, now contains all packages at versions that require PR fixes -
which will involve either bumping the patch version of an appropriate target
module, or removing the use of sys. Patching all packages in A would fix
100% of breakage, but would likely take too much time.

Now, given:

DLs(p)    = weekly number of downloads for package p
∂-(p)     = set of all dependents of package p
ownDLs(p) = DLs(p) - Σ(DLs(∂-(p))) – only the downloads originated by package p
P0 ➾͙ P1   = a potentially empty set of packages that represent the dependency chain
            between packages P0 and P1, non-inclusive
S(p)      = a tuple of (p, ownDLs(p)) for package p
Target    = target # of fixed weekly downloads

We run the following pseudocode:

list = []
for package in E ∪ A:
  list.push(S(package))

N = 0
fixablePackages = <empty set>
for package in sort(list, ownDownloadsDescending):
  N += ownDownloadsDescending
  for brokenPackage in B:
    fixablePackages = fixablePackages ∪ (package ➾͙  brokenPackage)
  if package in A:
    fixablePackages.add(package)
  if N >= Target:
    break

fixablePackages will now contain all of the packages necessary to fix the target mitigation
number of weekly downloads.

an aside

This is probably not the ideal way to collect this info! My maths are not the best.
Alternatively we could walk from each broken package along the dependent edge that introduces
the greatest number of weekly downloads, backtracking to the next largest source of weekly downloads
until we've met the target mitigation number.

Go / No go check: At this point there we should check to see if the amount of work
required to fully remove sys cleanly is achievable given the time frame.

step 3: go fix modules

Starting with the broken packages themselves, PR the appropriate fixes to packages. Expand to dependents as
new versions of broken modules are released. If a module is unmaintained, ask to take over.

Go / No go check: If v4.0.0 is looming at this point and we have not met our target mitigation number, the
we can't remove sys until v5.0.0.

[jasnell]

@chrisdickinson ... posts like that should come with automatic and mandatory free beer to help mitigate the pain. Great analysis, going to have to stew over that one for a bit...

[rlidwka]

Starting with the broken packages themselves, PR the appropriate fixes to packages. Expand to dependents as new versions of broken modules are released. If a module is unmaintained, ask to take over.

What happens if a module FOO which half of npm depends upon outright refuses to make any changes, claiming that the deprecation which TC agreed upon is unnecessary, or that util is a name of a god in their native language they may not write in code, or whatever else reason?

And they are hoping that node.js will never actually change it because it'll break too much stuff, thus using their amount of downloads to indefinitely stall a deprecation of a certain feature?

Kinda similar thing happened with url module in 2.0.0 release, so I'd like to establish a policy on those cases.

[Fishrock123]

What happens if a module FOO which half of npm depends upon outright refuses to make any changes

I'm not sure about the "outright refuses" part, but for modules where the maintainers no longer update them, we could always contact them / npm for access to publish a fixed version.

[rlidwka]

Yep, no longer maintained libraries are actually the simpler part. But actively maintained popular ones telling you "we don't want to change anything, our code works fine on 0.10.x and your new release is broken" is quite another matter.

Maybe I'll rephrase: what is the maximum period of deprecation after which the feature will be dropped no matter what?