Guide how to compile (AOT) of *.dart in Dart 2.0

[roman-vanesyan]

AOT compilation was possible on Dart 1.24 through CLI with using appropriated app-aot value for --snapshot-kind, but I've recently tried to compile *.dart source code on Dart 2.0 like this:

$ dart --snapshot-kind=app-aot --snapshot=app.snapshot example/http_handler.dart 

and it returned error with following context:

../../runtime/bin/main.cc: 251: error: Dart 2.0 AOT compilations only accept Kernel IR files as input ('example/http_handler.dart' is not a valid Kernel IR file).

Dumping native stack trace for thread 1428
  [0x0000560cff5767df] dart::Profiler::DumpStackTrace(void*)
  [0x0000560cff5767df] dart::Profiler::DumpStackTrace(void*)
  [0x0000560cff7b2302] dart::Assert::Fail(char const*, ...)
  [0x0000560cff1f3083] dart::bin::RunMainIsolate(char const*, dart::bin::CommandLineOptions*)
  [0x0000560cff1f3ae7] dart::bin::main(int, char**)
  [0x0000560cff1f43c9] Unknown symbol
-- End of DumpStackTrace
Aborted (core dumped)

Can you, please, provide a detailed guide how to AOT compile source as it seems Dart 2.0 isn't able to compile *.dart using CLI.

[roman-vanesyan]

May be, it's also a good idea to provide a complete documentation on dartlang.org how to AOT compile Dart code if it is possible in Dart 2.0 (I think it is possible, as Flutter uses it successfully)

[mraleph]

AOT compilation was possible on Dart 1.24 through CLI with using appropriated app-aot value for --snapshot-kind

I don't think this is true. In general you needed to use dart_bootstrap or gen_snapshot neither of which was part of the SDK.

Can you, please, provide a detailed guide how to AOT compile source as it seems Dart 2.0 isn't able to compile *.dart using CLI.

AOT compiler is not part of the SDK at the moment. If you have an SDK checkout you can do this:

# Build necessary components
$ tools/build.py -m release -a x64 runtime dart_precompiled_runtime

# Precompile
$ pkg/vm/tool/precompiler2 example/http_handler.dart app.snapshot

# Run the snapshot
$ pkg/vm/tool/dart_precompiled_runtime2 app.snapshot

Could you clarify why are you interested in AOT compilation? In general we would still expect JIT to reach better peak performance than AOT compiled code, though AOT code would have better startup latency.

[roman-vanesyan]

Could you clarify why are you interested in AOT compilation?

The purpose of my needs is just studying more about Dart as I'm new to it, to benchmark different modes of execution Dart support, understand capability of it, etc. I'd like to use Dart as primary language on the server side of the project.

In general we would still expect JIT to reach better peak performance than AOT compiled code, though AOT code would have better startup latency.

I believed that the static typing enforced in Dart 2.0 is used by AOT and generates a result code as a lot is known at compile-time and no runtime overhead etc.

[roman-vanesyan]

Yes, you are correct about this part:

I don't think this is true. In general you needed to use dart_bootstrap or gen_snapshot neither of which was part of the SDK.

I have just downloaded it and tried to run following was returned.

This VM was built without support for AOT compilation.

strange thing, I believed it worked.

[roman-vanesyan]

Are there any plans to provide general support for AOT in the distribution?

[mraleph]

I believed that the static typing enforced in Dart 2.0 is used by AOT and generates a result code as a lot is known at compile-time and no runtime overhead etc.

JIT has access the exactly the same type information as AOT - plus ability to apply profile guided optimizations and speculatively adapt the code to the workload - which often means that JIT actually knows more about actual types than AOT (though JIT has to pay for tracking it, where is AOT sometimes can run some global static analysis and compute it).

Just to give you a concrete (yet simplified and somewhat artificial) example:

void compute(List<int> l) {
  // do some computations with elements of the l.
}

void main(List<String> args) {
  final list = args.contains('something') ? <int>[0, 1, 2] : new Uint8List(3);
  for (var i = 0; i < 10000; i++)
    compute(list);
}

Here static types don't actually give AOT much information: knowing that something is a List<int> does not help much, as there are various implementations of List<int>.

JIT on the other side would specialize compute to a particular implementation of List<int> because it does type profiling in runtime.

Obvious in real world use cases the comparison is much more complicated, but in general you can't say that "AOT = fast, JIT = slow".

Are there any plans to provide general support for AOT in the distribution?

This is being discussed - but no immediate decisions have been made.

[corporateselect]

We would love to have aot compilation in distribution - use case: faas

aws lambda cold start with 1500MB Ram, Dart, Script snapshot: 950ms, App-JIT snapshot: 700ms, App-AOT snapshot: 70ms

[Indribell]

Any chance for a update on this issue. We are also looking to Dart 2 AOT compilation and the lacking inclusion of the AOT in the distribution is somewhat disturbing. As in "will this feature be supported in the future". Any update on this issue is welcome.

[mraleph]

@Indribell please include your use case here - that would help to prioritize the issue.

/cc @mit-mit

[Indribell]

@mraleph

We are a company that currently deals with micro services. Our uses cases involves internal and 3th party deployments. But we are looking to grow into the mobile and other market segments.

Here are some reasons:

• Restrictions from Clients. We have had clients who's had issues with languages like PHP, that dealt with code injections. The have very strict requirements like: "No open source projects", "no clear code deployments", "no ...". I agree that AOT is not a solution ( there are way to secure languages like PHP to prevent code injection ) but we are not in the business anymore to change client opinions anymore. Its the old "What the client wants, we provide" attitude.

• Micro service deployments. From our quick internal tests, Dart suffers from the same issue as most JIT languages ( .Net/Java/... ), memory allocation per service. Our designs for the back-end include up to hundreds of micro services per server and the default memory stack on each service its JIT is a wast of resources. JIT can be faster then AOT but it tends to never deal good with memory issues.

• Easier deployment. Note: Dart Snapshot deployment can do the same but exposes the code when doing so.

• Proprietary code. Some of our services are deployed on 3th party client servers, As you can imagine, we prefer not to have our code base open and public. We currently use Go to solve this issue. But as we have plans to expand, thus we are currently contemplating Dart because its interesting for having a single stack language combined with Flutter on all platforms ( Front, Back, Mobile ). Its easier on the developers and hiring process then having multiple languages being supporter internally.

For us Dart is a interesting project, as it combines the fast ability to use the JIT to develop and potentially using the AOT for deployment.

Please note that this is not a open discussion on preferences or why JIT or AOT is better for X or Y. These are our requirements as requested. For us its simply business decisions and not personal preferences.

We had the strong impression from Dart 1 articles and information scatter over the net, that Dart was AOT ready and distributed as such. It was surprising to find out during our testing, that Dart 2 ( we even tested 2.2 ) was not. In our experience when a language or project tends to back strip features from the default deployment, these features tend to get little support. This is why the original question got asked.

Color us surprised that the Web SDK did not include AOT.

Valid values are: none, kernel, app-jit

I hope this answer your questions.

[Rusticc]

@Indribell It seems several other programming languages, for example C, C++, Rust, would suit your use cases better. On the backend, they have unparalleled performance; on the frontend, they can be compiled to WebAssembly.

Flutter, currently, neither runs on the web nor can Dart be compiled to native code to run on servers.

[Indribell]

@Rusticc

Rust, C, C++ are languages that are too focused on low end system programming and as such require a different technology level of employees. The best way to describe C++/Rust is overkill for the tasks at hand.

After finding out that Dart had no focus on Back-end development, we switched over to Crystal ( crystal-lang.org ) and have been enjoying the experience.

[corporateselect]

@Indribell
You can compile to AOT if you check out the source and build it
And i dont agree that Dart has no focus on server - it only seems so i think cause i.e. of flutter gaining lots of attention... Dart for backend: very feature-rich and well maintained

@Rusticc
It's not about flutter - and dart does run on browser and it can be compiled to AOT to run near native speed... (And soon Flutter will run on web and every desktop) - but i agree, sometimes other languages are better suited - for us, we decided to take the huge advantage to have Dart for everything and live with some disadvantages

[Rusticc]

@corporateselect Dart AOT, currently, does not achieve anywhere "near native speed". It lags behind Java, for example.

And Dart is not the only option if you want to share code between backend and frontend. Go, for example, can be compiled to JavaScript with GopherJS. O'Caml with BuckleScript. Java with, among others, J2CL. The list goes on. All of these currently offer better AOT performance than Dart.

[lukepighetti]

Those who beat the Dart drum always say that it's JIT and AOT. I have been investing a small amount of time in taking Dart seriously for the server, with the end goal being to deploy to FaaS like Lambda. For this I knew I would need AOT to compete with alternatives like Node + TypeScript to keep the cold start times down.

The amount of messaging on AOT is so great that I was actually surprised that dart --build bin/main.dart didn't work. I started looking thru the docs and couldn't find anything. I was sure I was missing something. I found this issue and realized that I was not. Dart SDK does not support AOT out of the box from what I can tell.

Why the misalignment between messaging and delivery? I find it to be misleading. Is it because this capability is provided by community tooling?

[mit-mit]

Why the misalignment between messaging and delivery? I find it to be misleading.

Our intention was to message Dart's AOT support in context of the Flutter framework. Here the toolchain has fully integrated support for AOT compiling built into the flutter build command. I am sorry that our messaging here wasn't clear.

We currently do not expose this in the Dart command. I agree that this is a lost opportunity, and that something like server-less backends would greatly benefit from AOT-compiled Dart code that is able to spin up very quickly. I have even made some experiments myself (with a 'hacked Dart SDK') to confirm that this indeed does result in quick startup.

We hope to offer proper support in a Dart command in a future version; unfortunately I cannot offer a concrete timeline yet. Stay tuned to this bug for updates. Also please 'cast a vote' for this by hitting thumbs-up button on the top-most comment in this bug.

[Indribell]

@lukepighetti

The JIT and AOT messaging you see is Dart 1. Pay close attention to the article post dates and when Dart 2 came out. With Dart 2 it seems they did a total turnaround and switched to pure Client side focus ( this is even acknowledge in other issues tickets ). Another issues i advice you to take care off. Dart 2 JIT has fairly large performance / memory regressions going on compared to Dart 1 JIT.

Currently, if you want AOT, its up to you to manually compile this into the SDK. Flutter has the AOT for its supported platforms because they do this already. This is why a lot of AOT issues seems to be focused around Flutter related bugs.

My advice if you want a fast back-end, simply use Go or Crystal.

Crystal ( crystal-lang.org ) + Sentry ( https://github.com/samueleaton/sentry ) + Linux Parallel ( apt-get install parallel... if you run a lot of micro services and want to parallel run each sentry watcher ). This combination is a dream for developing and gives the same development feeling like Dart / PHP / ... But with 3 times more performance then Dart 2 JIT currently offers. A simple HTTP client in Dart eats close to 130 Megabyte where as the same client in Crystal uses barely 3 Megabyte memory. And the file watcher like Sentry compile in the background, so you can test ultra fast every change.

Dart simply runs into the same issue that Java ran into with its JIT. With the difference that Java has had a lot more time to develop their JIT performance.

[mraleph]

@Indribell If you have some concrete performance issues to report (e.g. "Dart 2 JIT has fairly large performance / memory regressions going on compared to Dart 1 JIT.", "But with 3 times more performance then Dart 2 JIT currently offers."), then I suggest you file separate issues for those.

We will certainly be happy to address those.

[lukepighetti]

@mit-mit thank you for the explanation, it was helpful to understand the current situation. I have always been of the opinion that Dart is now Flutter, but after playing around with server packages like aqueduct I am finding a lot of value in places I did not expect to find. Thing is, I am a big advocate of FaaS and was hoping to use Dart for it. Are you able to share your methods of AOT Dart on the server? Is it production ready? Thanks again.

[mit-mit]

Sorry, as I mentioned it's not supported yet, so I don't have anything to share just yet.

[corporateselect]

@lukepighetti we have dart with aot in production on aws lambda - we needed to build our own framework for this, but since aws lambda has layers it should be even easier to get it running. Overall we are very happy with this solution. As i mentioned Dart is a really good fit on server and very feature rich.

@mit-mit Thank you for clarification - Dart for server in general needs more public focus - it always feels like its dead, but the core libraries and other libraries are very well maintained - so it doesnt suit the public feeling

[mit-mit]

@corporateselect noted!

[lukepighetti]

@corporateselect if you find yourself with spare time I would love to know more about how you are AOT Dart 2 apps for lambda, since this issue is calling for documentation, perhaps that would resolve this issue? Thanks for taking the time to respond earlier.

[lukepighetti]

@corporateselect Did you find memory use improvement with AOT? My simple Aqueduct container is using ~250mb (routes + 3mb HashMap in memory cache). A similar Node container in my experience is usually around 70mb.

[mraleph]

@lukepighetti @corporateselect I suggest to have this dicussion outside of this issue, e.g. on [email protected] or on other mediums as it is unrelated to the topic of this issue.

[lukepighetti]

No problem, thanks for the reminder @mraleph

[ismlsmile]

I am searching how to aot dart code on desktop, and finally find this issuse. I think aot has many advantages: code protection, quick startup, less memory consumption, easy deployment. I am glad to see that aot on desktop is under consideration.

[diegopego]

I really need to have a single language on all development layers. From the server down to user interface! Good to see AOT for desktop being considered.

[swuecho]

https://medium.com/dartlang/announcing-dart-2-2-faster-native-code-support-for-set-literals-7e2ab19cc86d

read this from google news feed and get excited, finally end up being here.

at least give a note that dart sdk does not support aot out of box?

[lukepighetti]

Y'all need to sit down with marketing and get this ironed out. Just saying.

[wuya666]

Yeah, I was really excited when I saw that Dart official site says it can compile to native code and JavaScript and do all of backend server, desktop client, web, and mobile app (iOS/Android) development. And then I found out that it only compiles to native code for the mobile app part, not for backend server nor desktop client, what a bummer.

As the chief technical officer of a small IT service company (tech department has less than a dozen tech engineers) to provide commercial software services to clients, I'd really like to focus on a tech stack centered on one programming language instead of currently using Go for backend system, C# for desktop application, Vue for web and Java/Swift for mobile apps.

One thing I'm currently wondering is, if I build a snapshot of the app-jit kind, how hard is it to decompile it back into human readable source code? As a company that provides commercial software services, we definitely don't want our clients to get their hands on the source code.

[asaarnak]

@wuya666 Maybe this helps:

1. Is it possible to AOT compile strong mode dart on linux?
2. Vyacheslav Egorov https://groups.google.com/a/dartlang.org/d/msg/misc/BCfmQ6WTfNU/iNahT2YTBQAJ

It is possible to AOT compile (or precompile) *any* Dart on Linux/Windows/MacOS (on x64, ARM or ARM64 architectures) as long as your code does not use mirrors. 

Unfortunately you can't do this purely by using the SDK, because binaries you need are not distributed with it at the moment. 

You would need to build Dart from sources. Make sure to build dart_precompiled_runtime and dart_bootstrap targets. 

Then you simply do:

$ out/ReleaseX64/dart_bootstrap --snapshot-kind=app-aot --use-blobs --snapshot=app.snapshot app.dart 
$ out/ReleaseX64/dart_precompiled_runtime app.snapshot

Note: AOT compiled code has different performance characteristics from JIT compiled code, and at the moment might be considerably slower for
some specific use cases.

3. Florian Loitsch https://groups.google.com/a/dartlang.org/d/msg/misc/BCfmQ6WTfNU/decxYakdBQAJ

This might sound more complicated than it actually is.
Follow the instructions here to get a build on your machine:
https://github.com/dart-lang/sdk/wiki/Building#getting-the-source

Then do
tools/build.py --mode=release dart_precompiled_runtime dart_bootstrap

Then you should be able to continue with Slava's instructions.

Or this:

1. See how flutter does this:
   https://github.com/flutter/flutter/blob/802eca29d212856ba5cf98aae535dfbc982ef32a/packages/flutter_tools/lib/src/base/build.dart#L136
2. Tests in dart-sdk: https://github.com/dart-lang/sdk/blob/master/tests/standalone/app_snapshot_share_test.dart#L36

[mit-mit]

We're working on packing this up to make it more approachable. I can't offer a timeline, but we are making good progress!

[mit-mit]

cc @cskau-g

[ghost]

RE: asaarnak's comment above:
dart_bootstrap is no longer a target, but gen_snapshot can do much the same.
For easy use, anyone interested can have a look at pkg/vm/tool/precompiler2

[mnordine]

I see new binaries are listed in the changelog: 66fbaf3

[islishude]

try flowing

$ dart --version
Dart VM version: 2.3.1-dev.0.0 (Wed May 8 14:36:43 2019 +0200) on "macos_x64"
$ ls
test.dart
$ cat test.dart
void main() {
  print("hello,world");
}
$ dart2aot test.dart test.aot
$ ls
test.aot      test.aot.dill test.dart
$ dartaotruntime test.aot
hello,world

[ismlsmile]

version 2.3 has included dart2aot, and I tried, it is ok. But I am wondering if it can be combined into one exe file.

[mit-mit]

Yes, this is now supported as of Dart 2.3, and documentation has it covered here:
https://dart.dev/tutorials/server/get-started
https://dart.dev/tools/dart2aot

We don't support creating a single executable yet. I'm going to close the present bug; if someone wanted to create a request bug for single executable that would be great.

[asaarnak]

Thanks for working on it, really cool stuff!
This seems useful when using Cloud Run.
Tested with http server, works.

Tested performance with empty main method.
Ubuntu 19.10 with i7-8750H
In 10000 runs it took about 0,009s on avarage.

time for i in {1..10000}; do dartaotruntime main.dart.aot; done

real	1m29,327s
user	0m34,192s
sys	0m57,425s

[ghost]

@mit-mit

I am confused trying Dart AOT on my server.

I just installed Dart via linux setup (https://dart.dev/get-dart):

dart --version
Dart VM version: 2.3.2 (Unknown timestamp) on "linux_x64"

but there is no dart2aot nor dartaotruntime:

ls /usr/lib/dart/bin
dart  dart2js  dartanalyzer  dartdevc  dartdoc  dartfmt  pub  snapshots

uname -a
Linux 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:        18.04
Codename:       bionic

On my windows machine they are installed.

[mit-mit]

@flddr, maybe #36892 ?

[ghost]

Just want to say i have made very simple tests of short-time-running-JIT vs. AOT server side. Surprising is, every time AOT was ~10% faster, even if JIT runs some minutes.

Clearly, that's just funny here, but, it is ;)

copied from https://github.com/costajob/app-servers

import 'dart:async';
import 'dart:io';
import 'dart:isolate';

const String _HOST = '0.0.0.0';
const String _GREET = 'Hello World';

_startServer(arg) async {
  var server = await HttpServer.bind(_HOST, 80, shared: true);
  await for (HttpRequest request in server) {
    request.response
      ..write(_GREET)
      ..close();
  }
}

void main() {
  for (int i = 0; i < Platform.numberOfProcessors + 0; i++)
    Isolate.spawn(_startServer, null);
  _startServer(null);
}

JIT

** SIEGE 4.0.4
** Preparing 5 concurrent users for battle.
The server is now under siege...
Lifting the server siege...
Transactions:                  10712 hits
Availability:                 100.00 %
Elapsed time:                   4.07 secs
Data transferred:               0.11 MB
Response time:                  0.00 secs
Transaction rate:            2631.94 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                    4.91
Successful transactions:       10712
Failed transactions:               0
Longest transaction:            0.03
Shortest transaction:           0.00

AOT

** SIEGE 4.0.4
** Preparing 5 concurrent users for battle.
The server is now under siege...
Lifting the server siege...
Transactions:                  12404 hits
Availability:                 100.00 %
Elapsed time:                   4.12 secs
Data transferred:               0.13 MB
Response time:                  0.00 secs
Transaction rate:            3010.68 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                    4.94
Successful transactions:       12404
Failed transactions:               0
Longest transaction:            0.02
Shortest transaction:           0.00

That's running on a virtual 1-core. I have tested several snippets, this is just the last one ;)

For sure, on a bigger machine in real-world, JIT will have advantages over AOT, but, for many small services etc. this is an excellent addition to us 😊 ❤️

[dzpt]

@flddr @mit-mit can it import and run with binary package?