Feature Request: a way for a Dart library to refer to resources that are not Dart code

[DartBot]

This issue was originally filed by @sethladd

---

Use case: a library needs to retrieve files inside itself and copy them to the current working directory. These files are not Dart code, instead they are text files and binary images. This library should work on the command line or as a web app or Chrome App. (that is, we can't always assume dart:io is available and we can't assume that Platform.script returns a file: URI)

Current workaround is to embed the files' content into .dart files as triple quoted strings of text or base64. This is less than ideal:

* It requires some sort of build step for the library

• dart2js includes all of this content in its output, greatly increasing the size of the downloaded JS

[lrhn]

I believe this requires some sort of language assistance.

We could have an Isolate.loadResource(uri), but that wouldn't give a way to find the URI of a file relative to the current library.
A library doesn't know its own URI, so it can't create a URI referencing something relative to itself.
The load function doesn't know which library it's being loaded from, so it can't resolve a relative URI depending on who is loading.

One possible way to add the current library's URI:
  library foo.bar.bas as foo;
where foo then resolves to the URI of the library.

Then we can add Isolate.loadResource(uri) which must accept URIs including package:-URIs.

---

Removed Type-Defect label.
Added Type-Enhancement label.

[DartBot]

This comment was originally written by @sethladd

---

Note to anyone watching: please add your use cases here.

Our specific use case is a project generator. The template files are logically part of the library. The code in the library wants to load up the templates, do some search and replace, and write files to disk or output the files (if only we had a platform neutral file API! :)

[DartBot]

This comment was originally written by @seaneagan

---

I would like to be able to load the contents of a package's pubspec.yaml for this feature:

seaneagan/unscripted#21

Originally filed issue #19434 for this, but maybe this is a better option.

[lrhn]

I don't think the pubspec.yaml file is generally available at runtime. It's not necessary for deploying a library as part of another application, it's only used while creating the deployment files.

[floitschG]

Added Area-Library, Triaged labels.

[DartBot]

This comment was originally written by @seaneagan

---

Correct, pubspec.yaml is not currently available at runtime. In npm (JavaScript package manager), it's quite easy to do:

http://stackoverflow.com/questions/9153571/is-there-a-way-to-get-version-from-package-json-in-nodejs-code

I don't necessarily need access to pubspec.yaml via it's original file system location at runtime... I just need access to the package-local pubspec.yaml's contents at the runtime produced by "pub serve/build/run", which means I'd be willing to require use of a transformer to get this if necessary, but that was rejected in issue #19434. It'd be best if it also transparently worked when not using "pub serve/build/run".

[devoncarew]

I think it'd be nice to have a programatic way to reason about the packages in the system. So, something like:

Package package = Package.current;
print(package.version);
print(package.dependencies);

or

List packages = Package.all;

or

Package.current.getResource('foo_bar.dat');

[DartBot]

This comment was originally written by @seaneagan

---

I guess that API would be a PackageMirror class in dart:mirrors. A LibraryMirror could have a package field to access it's PackageMirror, or something.

But there would need to be some way to preserve the package metadata when this API is being used, and tree shake otherwise or something, and would have to decide which metadata to expose... obviously the package name, then version makes pretty good sense, not sure anything else from the pubspec really does though other than maybe dependencies.

[DartBot]

This comment was originally written by @seaneagan

---

Just wanted to point out that technically you could write a transformer to transform a non-dart resource into a dart file with a string variable containing the content, and then use deferred loading to load it, and that would also work cross-platform. To implement the deferred loading, you'd probably want to write an AggregateTransform:

https://www.dartlang.org/tools/pub/transformers/aggregate.html

which outputs a dart file with a method which takes a URI and loads the correct resource using a Map from URI to library id or something, and returns a Future for the content.

Hmmm, this sounds like a fun project, I might try it!

[devoncarew]

Yeah, we're using some build-time steps to do just that - concatenate non-dart resources into a base64 string in a dart file (https://github.com/google/stagehand/blob/master/tool/grind.dart#L113). It has the drawback that the compiled size of the application increases. Deferred loading would mitigate that somewhat. It would be nice to just have access to the resources.

[DartBot]

This comment was originally written by @kaendfinger

---

This is a great idea!

[lrhn]

How about having a platform (server/browser) specific way to load resources.
Deployment would then have to put the resource files from all packages into a central location that the platform can find, and then you can load them from there using some path that the package can know.

Example: The package foobar has a resource directory called "resources", and some way on the yaml file to specify that this is its root resource directory.
When deploying, all the files are moved to "foobar/" in the deployment resource directory.
Inside the library, you can refer to resource as const Resource("foobar/img.png"), which gives you one or more ways to load that resource. The VM resolves it like it does package URIs, by looking at it relative to a root resource URI, similar for the browser.
I guess "pub server" can redirect a HTTP resource root to the original files without copying.

(The constructor is const, so you can define the resource objects as constants, and even have an offline check that all the resources exist).

[munificent]

How about having a platform (server/browser) specific way to load resources.
Deployment would then have to put the resource files from all packages into a central location that the platform can find, and then you can load them from there using some path that the package can know.

That was basically our plan with bartender:

https://docs.google.com/document/d/1VEtbHCW-XT1OLIwS8F_oICnDrH6YPzKVbuSDph2PIOY/edit

Barback was then sort of the back end for that. Alas, we haven't yet gotten to the point where we can implement bartender itself. And, given the lack of platform-specific code, it isn't possible to implement it.

[sethladd]

dartdoc could really use this.

When you install dartdoc via pub global activate, it generates a snapshot and puts that snapshot outside of the package location. The code can no longer reliably find any resources (text files, images, etc) which may be required for the app to run.

There's no API for me to associate a package's resources with a package's script, especially if that script is actually a snapshot in some random location on disk.

The workaround is to encode every resource (even binary resources) as strings and insert those strings into a .dart source file. This is what grinder does. This is less than ideal: a build step is required before I can run my tool during development. This creates friction for the developer, and slows down the dev cycle.

There are a few potential solutions to this issue. Probably a combination of a new resource loading API and changes to how a pub package is globally installed.

Would really help to have a solution in time for 1.11 so we can roll out dartdoc with grace. :)

Thanks!

---

cc @lrhn.
cc @iposva-google.
Removed Priority-Unassigned label.
Added Priority-Medium label.

[DartBot]

This comment was originally written by @kaendfinger

---

I'm going to see if I can do a prototype of the Uri.loadAsString idea. I like this, and it seems like a good way to go.

[DartBot]

This comment was originally written by @sethladd

---

Did one this weekend, though I'm sure it has tons of bugs: https://github.com/dart-lang/dartdoc/blob/warning-on-snapshot/lib/resource_loader.dart

It compensates for a lot of assumptions, though.

[DartBot]

This comment was originally written by @kaendfinger

---

If we could extend the snapshot system to allow resources and things like native extensions, that would be great.

[DartBot]

This comment was originally written by @sethladd

---

Snapshots are tied to a very specific VM version. They currently aren't an option for a long-lived deployment bundle.

[DartBot]

This comment was originally written by @kaendfinger

---

Ahh, good point, forgot about that. A bundle would contain source code then. Would it be in an archive format?

[sethladd]

In progress. @lrhn has a CL.

[sethladd]

CL is up:
https://codereview.chromium.org/1181663002/

[lrhn]

The Resource class has landed.

[sethladd]

👍