New model (for unified score and more)

[isoos]

The discussion around the dartdoc score identified the need for a new model on how we store the score(s). Until now we have stored it in PackageVersionAnalysis, which is tightly-coupled with storing the result and serialization of pana.

The model also introduced some operational complexity, as we built the UI around the internals of pana, and changes to it require consideration on how we roll out a release. We've also developed a few things to cache the quick summary of the analysis (AnalysisExtract) in order to display it along the various UI places.

Considering the experiences on how we use the data, and a few things that are in the pipeline, I'd suggest the following:

• Let's introduce a Datastore entity for storing the high-level analysis data (essentially AnalysisExtract + the new dartdoc score), but decoupled from the pana-related work.

• Let's introduce a Datastore entity and a pub-site-only data model for the output of pana and dartdoc analysis. This should contain the information on the analysis tab, plus the collected suggestions from both pana and dartdoc. For each new serialization change of pana we shall transform it to the pub site internals. This would add a bit of work when there is a breaking change in pana, but it would reduce the deploy worries and complexities.

• If/when we'll have a facility for runtime analysis, it would be easy to adapt its result into the above items.

As far as I can tell, this should be possible in an incremental way, and we may do a full migration in a couple of deploys.

/cc @sortie @kevmoo @mit-mit

[isoos]

It looks like that the model can't be as simple as I've originally suggested, because it lacks the ability of handling the rollback of a partial rollout. We'll need to extend it with the runtimeVersion as part of the key, but at that point it could also merge with the current job processing (which has the same issues with the rollback, for the very same reasons).

The updated suggestion:

• ScoreCard and their detailing ScoreCardReport ids should be in the form of package/version/runtimeVersion.
• Each deployed runtime should keep track of or if needed update their versioned entries, independent of other runtimes. That way if there is a rollback, the new entries created by the new runtime will never interfere with the old ones.
• A GC process could periodically scan the table and remove entries that were not updated for more then a few months.
• Each deployed runtime should try to load their own version first, and if needed, fallback to earlier runtime versions, but never load later runtime versions.

[isoos]

Another aspect of mixing this approach with the current Job processing is the coordination. The Job model requires frequent Datastore scans and individual update of the entries, mostly for (1) figuring out what to do and (2) preventing race conditions. The new model with the runtimeVersion would able to simplify (1), and because we could use memcache for coordinating (2), it could be a big win in terms of overall complexity.

The only drawback I can see now is the handling of the flags (e.g. discontinued), because they are package-level, and we may need to re-introduce the old version of the task notification system.

[isoos]

This is now in prod for a while.