chore: base setup for use of API doc code-excerpt updating tools

[chalin]

This PR illustrates how to use two Dart tools to update code excerpts in API docs. The hope is that these tools can be used both internally and externally.

TL;DR

---

The two tools used to refresh code excerpts in the site-webdev and site-www docs are now written in Dart:

• https://github.com/chalin/code_excerpter is written as a builder. It scans source files for #docregion markers and creates *.excerpt.yaml excerpt files from the named regions.
• https://github.com/chalin/code_excerpt_updater can be run using pub run. Using the generated excerpt files, this tool scans the docs, refreshing any outdates excerpts as needed.

This PR illustrates the base setup for use of the tools. The tools are simply listed as dev_dependencies in the top-level pubspec.yaml.

This is how you can generate the code-excerpts from sources (both the local examples/** and the site-webdev example apps):

pub upgrade 
pub run build_runner build --delete-conflicting-outputs --config excerpt --output tmp/excerpt

I've had to configure the builder to run over all packages, so the first build takes 22s (because some of the site-webdev example apps use ACX), but its 2s after that.

To update code excerpts in API docs, you can run commands like the following for angular_forms:

pub run code_excerpt_updater \
  --no-escape-ng-interpolation \
  --fail-on-refresh \
  --write-in-place \
  --src-dir-path tmp/excerpt \
  --fragment-dir-path tmp/excerpt \
  --yaml angular_forms

You can update entire packages or individual files/folders:

pub run code_excerpt_updater --no-escape-ng-interpolation --fail-on-refresh --write-in-place --src-dir-path tmp/excerpt --fragment-dir-path tmp/excerpt --yaml angular/lib/src/common/directives/ng_if.dart

I've run both of these commands and included the resulting changes in this PR.

Hopefully these two tools could be used both internally and externally, potentially contributing to solutions for the following issues:

• Validate that code snippets in docs are at least "analyzer clean" Validate that code snippets in docs are at least "analyzer clean" #996
• Test & update/remove the examples in the API docs Test & update/remove the examples in the API docs #1267
• Should docs and examples for webdev.dartlang live here? Should docs and examples for webdev.dartlang live here? #942
• API docs: OnInit has inlined code and no longer links to Lifecycle docs API docs: OnInit has inlined code and no longer links to Lifecycle docs #602
• Update documentation for Query.read to include an example Update documentation for Query.read to include an example #249

cc @alorenzen @matanlurey @kwalrath @kevmoo

[chalin]

This illustrates use of code from examples/hacker_news_pwa.

[chalin]

List site-webdev external apps here.

[alorenzen]

@chalin thanks for sharing these. We have a docs fixit schedule for next week, where we're hoping to spend some time collating and improving our internal docs. After that, we should have a much better understanding of if / where we would want to use these tools.

[chalin]

Any update regarding this? /cc @filiph

[alorenzen]

Thinking through how this would work in our internal/external workflow, here's what I've got: we basically treat examples like we do our golden files.

We can use code_excerpter as a test to tell us when the examples need to be updated. Since it is a builder, we can use the exact same approach as we did for the goldens.

We would then use code_excerpt_updater as the update script. We'd need a separate internal / external version like we have now for the goldens, but that shouldn't be too hard to handle.

We wouldn't use the "live snippet" feature of our internal markdown viewer, but this would still allow us to have our examples as actual examples that can be built / analyzed, and then embed them in the docs for both internal and external users.

@matanlurey does this sound reasonable to you?

I think an interesting question at this point is if we want to fork the tools and have them live in the angular repo, or if we want to make them official tools under dart-lang and just have wrappers in angular.
@matanlurey @kevmoo @kwalrath any thoughts on this?

[kevmoo]

I think an interesting question at this point is if we want to fork the tools and have them live in the angular repo, or if we want to make them official tools under dart-lang and just have wrappers in angular.

Could we start w/ these tools being in the angular repo and then graduate them to their own package/repo if we find other interested users?

[chalin]

Could we start w/ these tools being in the angular repo and then graduate them to their own package/repo if we find other interested users?

FYI, the tools are currently used by dart-lang/site-www as well as dart-lang/site-webdev. @filiph is considering their use elsewhere too.

[kevmoo]

Then we've passed my threshold for creating a repo: used by 2+ projects, has clear owner, plan to keep using to the horizon

[matanlurey]

All of my thoughts below. We can schedule a meeting or start a doc to review further:

@alorenzen:

Thinking through how this would work in our internal/external workflow, here's what I've got: we basically treat examples like we do our golden files.

I like what I'm hearing, and I think it's reasonable (your ideas, that is).

@kevmoo:

Could we start w/ these tools being in the angular repo

@chalin:

FYI, the tools are currently used by dart-lang/site-www as well as dart-lang/site-webdev.

Unfortunately I don't feel comfortable using tools that we don't have direct control over. We've been burned and slowed down too many times in the past. If they lived in the AngularDart repo, or we have direct control over what they do, at least until we have tests and integration in place, I'd be OK with making them more open/all over the place.

(Some historic examples: Barback, MonoRepo)

The reality right now is we need a way to be directly involved in keeping our examples and docs up to date, and not have to go through external folks or tools. However we can accomplish that is fine with me (and @alorenzen seems to have the right ideas, so I'll defer to him).

[alorenzen]

@chalin

Could we start w/ these tools being in the angular repo and then graduate them to their own package/repo if we find other interested users?

FYI, the tools are currently used by dart-lang/site-www as well as dart-lang/site-webdev. @filiph is considering there use elsewhere too.

If we're going to use them internally, then I don't think the tools should live under https://github.com/chalin/

@kevmoo

Then we've passed my threshold for creating a repo: used by 2+ projects, has clear owner, plan to keep using to the horizon

I don't want individual owners. I want a team responsible for it. A team with OKRs that we can influence.

[chalin]

If we're going to use them internally, then I don't think the tools should live under https://github.com/chalin/

Totally agree; dart-lang seems like a good choice.

[matanlurey]

I am going to close this because I think this PR will be stalled a bit until we have time for it.

(And we are still working out the details offline)