How to support "deprecation" and "intervention" use cases

[RByers]

@patrickkettner @igrigorik @mikewest and I have been chatting offline about using the Reporting API to report deprecation warnings and warnings about interventions being hit.

Let's move the discussion here for somewhere public. Presumably this will turn into either a new WICG repo or perhaps just an additional specification inside this repo (or maybe I'm wrong and no knew spec text is required at all?).

Here's a summary of the key questions we've been discussing so far:

Do we need an API for these warnings

It sounded like @patrickkettner was thinking we could get away without any new API surface area. I think we're going to want developers to be able to explicitly specify the reporting group and enable/disabling reporting for deprecations and interventions. But maybe that's not necessary in an MVP?

Can deprecation/intervention reporting be on-by-default

@igrigorik said:

FWIW, the spec already has the right bits in place for "default": http://wicg.github.io/reporting/#id-member -- specifying a group name is optional when the endpoint is registered. From the other end, we could queue all warnings to default group and provide a meta+header override to toggle such reports on/off and specify the target group.

What should the name of the header be to control this

@igrigorik said:

Warnings: on; report-to=groupname.. or some such.

Virtually all Reporting API use cases are "warnings", right? So would this be a catch-all for report types that aren't explicitly configured elsewhere? Should we consider (now or in the future) some ability to filter by "type". Eg. Warnings: deprecation; report-to=groupname.

Or is it better to just have report-type-specific headers like Deprecations and Interventions?

How much of the report format for deprecations and interventions should we specify?

For deprecations I can imagine implementing something like this:

{
  type: "deprecation",
  age: 10,
  url: "https://example.com/",
  report: { 
    id: "websql", 
    anticipatedRemoval: "1/1/2020", 
    message: "WebSQL is deprecated and will be removed in Chrome 97 around January 2020",
    sourceFile: "https://foo.com/index.js",
    lineNumber: 1234 }
}

In particular, I assume we'd want a stable (but impl-defined) id to enable services to group reports for counting/reporting purposes, and some sort of mechanism for prioritization.

@igrigorik suggested a type of simply warning. But would something like that be too generic to be useful? All reports are warnings, right? Is the schema for report intended to be determined by the type? I.e. if we want structured information like anticipatedRemoval then we'd want to distinguish deprecations from interventions in the type, right?

Are there any potential consumers we should engage with on the design?

Google Analytics is an obvious option. @igrigorik are there any folks from any of the RUM vendors who would be useful to bring into this design discussion early?

[igrigorik]

@igrigorik are there any folks from any of the RUM vendors who would be useful to bring into this design discussion early?

/cc @nicjansma @n8schloss :)

[n8schloss]

@RByers everything you wrote above sounds reasonable to me, I think we'd be pretty supportive of something like this!

[nicjansma]

👍 I like the idea a lot.

The Reporting API is only able to send reports to HTTP endpoints specified via the page's HTTP response headers (e.g. Report-To), correct?

As a RUM vendor, it's generally somewhat of a pain (or takes a long time, or is a non-starter) to get things like HTTP headers added to our customer's websites. We can get our third-party script tag on their sites, but that's about it.

Along those lines, since our script is already living on our customer's pages, we've recently started collecting JS errors (window.onerror etc). Deprecation warnings would be a perfect addition to capture/report on as well, but it would be a lot easier for us if there was a JS API (in addition to the Reporting API) to consume deprecation notices.

One big difference with the Reporting API (as I understand it) is that it's goal is to be able to report when a page can't even load -- whereas deprecations would be the opposite. So I'm not sure if deprecations fit within the Reporting API, unless it could be extended to also report via JS (can I get a ReportingObserver?).

[RByers]

@nicjansma thanks! Those are problems we've discussed in other reporting scenarios as well - filed #28 and #29 to discuss some ideas.

I chatted some more about this with @mikewest and @patrickkettner at BlinkOn. I think we agree that we should start with a "basic reports" explainer (eventually spec) here in the reporting repo that defines the use cases and report formats.

@mikewest says we'll probably want to add a note to the main spec saying that some report types can be enabled by default to go to the default group (and it's probably fine if we don't even provide an explicit API for enabling/disabling or setting the reporting group for the basic reports (at least in the initial MVP).

[RByers]

I've written an initial rough explainer for this (and reporting generally). Thoughts?

[RByers]

#28 suggests that we should have a completely independent DOM event for deprecation (and probably also intervention) warnings the way CSP does. That seems reasonable to me (eg. a single "UsageWarningEvent" fired at the window).

In fact, if we're going to do that anyway, then it's not clear to me how much additional value there is to also having the Reporting API support. I suppose the reporting API is always going to be more reliable, and may be more convenient to enable in some cases. I'd be happy shipping the DOM event first though. Thoughts?

[igrigorik]

If we plan exposing a growing number of these, having a common event/mechanism to subscribe from JS would be nice. Maybe that's the connection to Reporting.. or maybe not. I do agree that the background delivery part of the process is not a strict requirement for deprecations and warnings.

[RByers]

I guess the thing that's really valuable to have common infrastructure for is the background delivery.

There are a few properties on SecurityPolicyViolationEvent which I could see potentially moving into a common ReportingEvent base class ('documentURI, sourceFile, lineNumber`) to potentially make it easier for a RUM vendor to treat all reporting events with the same code (just have a list of event names, and JSON-serialize the event object to get all the details up to the server). But I'm not sure if there's really enough value in that to justify, WDYT?

Regardless it sounds like we should have some DOM event for deprecation and intervention cases. @patrickkettner, do you agree? If so I'll create a new repo (eg. "API Usage Warnings") and write an explainer for that (and have it hook into reporting the way CSP does). Sound good?

[patrickkettner]

cc @jdalton @ScottHelme

everything sounds great to me. The only open question in my mind is how we would specify things in id that do not have direct APIs (like "we will start to show control characters (and you have some in your code)", or changing the arity of a function, etc).

it sounded like @patrickkettner was thinking we could get away without any new API surface area. I think we're going to want developers to be able to explicitly specify the reporting group and enable/disabling reporting for deprecations and interventions.

This comes down to what we want the "default" group to be. Considering how hard it is to message deprecations to developers, I would prefer that the simplest settings (i.e. no group specified) include them.

Regardless it sounds like we should have some DOM event for deprecation and intervention cases. @patrickkettner, do you agree?

an event unique to each type, or a generic report event? This is just #29, correct?

[ScottHelme]

Thanks for the cc @patrickkettner.

The timing of this is uncanny. I've been working on a separate site/service and one of the features was going to be how to warn site operators about upcoming feature deprecation. The example is usually detecting use of the geolocation API on http and having warned before it broke without warning.

This would be a great fit for me on @report-uri and I'd be happy to help how I can.

[mikewest]

@RByers: Over time, I'd be happy to move things off SecurityPolicyViolationEvent onto a generic reporting event. If you need help with a document that makes that explicit, I'm happy to help.

I think I'd actually suggest being even more generic: ReportEvent, with a type that developers can hook into, seems like a better fit for the mental model of the Reporting API in general than several specific reporting events that all inherit from a generic event type. But that's bikeshedding: I'm happy either way.

[patrickkettner]

ReportEvent, with a type that developers can hook into
👍

[RByers]

@paulmeyer90 has added deprecation and intervention reports to the spec, and Chrome is shipping deprecation reports.

Probably all that's left to close out this issue is to get web-platform-tests for deprecation and intervention reports (which has been blocked on some WebDriver work for automation). @paulmeyer90 are there issues / bugs elsewhere tracking that work?