🪝 [breaking change] Reserve top-level `hook/` directory 🪝

[dcharkes]

Change intent

(edit: Updated 2024-01-24 to reflect use of the hook/ toplevel directory.)

A package toplevel hook/ directory will be reserved for scripts run by the Dart and Flutter SDK at specific lifecycle events with a specific commandline API.

Examples:

• hook/build.dart will be run before kernel compilation of Dart packages.
• hook/link.dart will be run after kernel compilation of Dart packages, before making an application bundle.

If these scripts exist, they will be invoked by the Dart and Flutter build systems.

If these scripts exist, they must implement the commandline API.

For more context:

• ☂️ [vm/ffi] Build hooks & Code assets #50565

For more

Justification

Other languages standardize on a top level build script that builds code in other languages.

Most notably Rust defines a top level build.rs in packages to build native non-Rust code.

• https://doc.rust-lang.org/cargo/reference/build-scripts.html

Reasoning for using hook/build.dart and hook/link.dart in Dart.

• A Dart package public surface is the lib/ and bin/ directory. But build scripts should not be part of the programmatic or CLI API of a package.
• A package private scripts reside in tool/. But build scripts should be run from the context outside that package itself.
• The build/ directory is already reserved in Flutter. So nesting the build script inside the build/ directory doesn't work.

Alternatives considered

• toplevel build.dart and link.dart. Rejected because we might want more scripts.
• toplevel build_assets.dart and link_assets.dart. Rejected because of longer names.
• [native_assets][ffi] Make build.dart path configurable #54111

Impact

Anyone using a top level hook/ directory will need to migrate to a different file name.

We are aware of one use:

• [BUG] Build fails on Flutter 3.16 for iOS or Windows epam-cross-platform-lab/swagger-dart-code-generator#681

Mitigation

Migrate existing uses of hook/ to tool/, which is probably what those scripts are.

Action items after accepting proposal:

• Update https://dart.dev/tools/pub/package-layout to include hook/
• [native_assets_builder] Rename build.dart to hook/build.dart native#823
• [native_assets] Use hook/build.dart instead of build.dart flutter/flutter#138727

[dcharkes]

Ready for another round of bike shedding. @mit-mit @mraleph @mkustermann @mosuem @devoncarew @HosseinYousefi @jonasfj and who else wants to chime in.

[HosseinYousefi]

Another solution would be to introduce a top-level directory called assets where build.dart and link.dart reside in.

[dcharkes]

Having a directory would fit with a possibility of having pub resolve dependencies differently in a directory:

• [new feature] build_dependencies pub#3794

Another solution would be to introduce a top-level directory called assets where build.dart and link.dart reside in.

Flutter already has an assets/ directory as well. Maybe it's a good thing to have Dart scripts in there, maybe it's a bad thing because the files in that directory are currently interpreted as non-executable and bundled as is? (https://docs.flutter.dev/ui/assets/assets-and-images)

[HosseinYousefi]

Having a directory would fit with a possibility of having pub resolve dependencies differently in a directory:

• [new feature] build_dependencies pub#3794

Yes! asset_dependencies.

Another solution would be to introduce a top-level directory called assets where build.dart and link.dart reside in.

Flutter already has an assets/ directory as well. Maybe it's a good thing to have Dart scripts in there, maybe it's a bad thing because the files in that directory are currently interpreted as non-executable and bundled as is? (https://docs.flutter.dev/ui/assets/assets-and-images)

The place for assets is not hardcoded in Flutter, but it's true that usually people use the name assets for the top-level directory containing their icons, fonts, ... . I don't think that they include any Dart scripts there, we could show a warning/error if they accidentally include dart files as assets (especially for build.dart and link.dart).

[lrhn]

I'd also prefer to not such scripts in the root directory.

Top level scripts are not part of the public surface of a package.

And with good reason, it's because they shouldn't be. That's prime real-estate that should belong to the author. Or for the author to ensure it's kept clean, if they don't want clutter.
(I'd also move pubspec.lock into .dart_tool if we started creating it today.)

The tool directory already exists, otherwise it would make sense (or tools, but having both would be very confusing).

Probably not a .name, otherwise I'd go for .build. Hmm, maybe it can work. But not if some people are blanket-git-ignoring .*, which they might.

Maybe scripts? build_scripts? build_tools?

Do we expect more scripts to be added in the future, or is this going to be about FFI native assets forever.
Maybe some scripts can be triggered by macros (say a macro which requires downloading and caching some specification, so it can depend on that while running).

If so, the script names should be less generic. Maybe asset_build.dart and asset_link.dart.

Is "asset" the best name, if that's already what people are using for their icons? (The name has always confused me when used about native resources, probably because I associate it strongly with web assets.)

Could it be ffi_build.dart, ffi_link.dart? That's specific enough to be precise. I might even understand what it is :)

[dcharkes]

Could it be ffi_build.dart, ffi_link.dart? That's specific enough to be precise. I might even understand what it is :)

@mosuem wants to use the (to be built) link.dart to trim/tree-shake json files for internationalization/translation. So this is not FFI related.

Whatever comes out of build.dart and link.dart will end up in the final application bundle. And it's not compiled Dart code (most likely).

That being said, maybe we have a need for some kind of pre-defined script in the Dart/Flutter eco system that is not related to bundling assets (be it code or data), but is a predefined script name. So in that case even "asset" is not the right name either. Currently "asset" is the right name, but it might not be in the future.

Maybe scripts? build_scripts? build_tools?

I think I like the last one best, but it's long.

Python and Node use the term "hooks". Som we could reserve the folder hooks? hooks/build.dart. (But then "hooks" sound a lot like an implementation detail, like a "callback".)

[lrhn]

Will the scripts need to be dart files, or could we allow someone writing a build.sh or build.py instead?
We pobably wouldn't allow publishing that, since it's platform dependent. So I guess they can write a build.dart which exec's python build.py.

[dcharkes]

We require these scripts to be dart scripts. We can use the "current" Dart executable to run these scripts from where we want to run them. We could require the dart scripts to be marked executable and having dart on the path, but that means we cannot rely on that Dart executable being the same one as "the host". So I think that would be a bad idea.

So I guess they can write a build.dart which exec's python build.py.

Be my guest. 😅

[ykmnkmi]

Do current build stack and web compilers migrate to this? I love to see something like zig build system. Typed. No yaml files.