[native_assets_cli] Proposal for versioning of core CLI build protocol and protocol additions

[mkustermann]

At the core of it the CLI build protocol is the ability of a package to tell how app bundling tools (e.g. Flutter SDK, Dart SDK, ...) about various assets it needs at runtime.

Once we have a thriving ecosystem of packages making use of this ability there's a very high bar on keeping the protocol between the app bundling tools and the package's hook stable. Though there may come a time when we have to change the protocol. If we do so, we have to minimize any negative impact this can have on the ecosystem.

The first mechanism is to decouple the protocol into core protocol and protocol extensions (e.g. data assets, code assets). This means a breaking change to one protocol extension limits the possible negative affects to only the packages that use that protocol extension and not others.

How would we approach breaking the core protocol or a protocol extension? We should avoid situations where publishing new app bundling tools (Dart/Flutter/... SDKs) breaks users in a way that's very non-obvious (e.g. updating SDK leads to some dart run / dart build failing and errors are burried deep in the --verbose log). Ideally users would be told: You updated Dart SDK, which comes with breaking change, but package Y wasn't prepared for this breaking change.

Currently the CLI protocol between bundling tool and hook communicates a version number. Though this version number is problematic due to several factors: It will make hook runs fail and errors will be buried deep in logs, the version doesn't cover protocol extensions, ...

I propose us to use the existing mechanism we have for packages for breaking changes: Use package's semantic versioning system. That will make users get nice error messages if packages & SDKs are not compatible. It also allows us to soften the blow of any breaking changes in a way that may be invisible for end users. The idea being roughly as follows:

• Every protocol piece (core protocol, protocol extensions) get their own dart package
• Those packages will get semantic versions for the APIs they surface
• Those packages will get SDK constraints (possibly also Flutter SDK constraints - if flutter specific protocol) as follows
  • We publish them with narrow upper bound constraints: sdk: min < X < max such that
    • min is the minimum SDK where the supported features of the protocol work
    • max is the current SDK versions plus 1
      => We promise that the next SDK will continue to support this version of the protocol package
  • Most common case: no breaking changes, but new SDK release: We'll publish new protocol packages with increased upper bound
  • If we do a breaking change
    • We always have a transition period
    • We release a SDK with the new thing, with protocol packages with a higher minimum SDK constraint
    • At this point packages can start adopting the newer API, but the new Dart SDK still supports the old protocol package-version
    • Support packages (e.g. package:native_toolchain_c, package:data_assets, ...) may publish minor version updates adopting the new protocol (requiring the new protocol version package)
    • We release another SDK at which point the old protocol package-versions published will no longer work (as their upper bound is lower than newer SDK release)

For end users that use

• older SDKs, pub will automatically select older support packages which use older protocol versions
• newer SDKs, pub will automatically select newer support packages with newer protocol versions
  => Overall end users may not even know that breaking changes have appeared, by pub selecting the right packages for the SDK in use.

The same mechanism can also be used to version our dart_api_dl.h: It can be published as package:dart_api_dl, use narrow upper bounds with promised transition period for any breaking changes.

(In some sense this system would allow versioning sub-parts of our SDK and softening the blow to any breaking changes by making pub auto-select SDK-wrapping packages that work with the currently used SDK)

[dcharkes]

Duplicate #93.