Native executables created with dart2native do not support signing

[mit-mit]

NOTE: This issue specifically refers to dart2native tool, it does not refer to Flutter on Desktop, as Flutter does not use dart2native

The executables created with bin/dart2native use a format that as discussed in the original issue is not compatible with signing tools such as codesign and signtool.

[anaisbetts]

Confirmed this on Windows 10 - signing a binary with signtool on Windows will cause it (strangely enough) to act like dart.exe

C:\Users\ana\code> cat .\test.dart
main() {
        print("Hello world!");
}

C:\Users\ana\code> dart2native -k exe -o test.exe .\test.dart
Generated: c:\users\ana\code\test.exe
C:\Users\ana\code> .\test.exe
Hello world!

C:\Users\ana\code>signtool.exe sign /f mycert.pfx /p sekritPassword .\test.exe
Done Adding Additional Store
Successfully signed: .\test.exe

C:\Users\ana\code> .\test.exe
Usage: dart [<vm-flags>] <dart-script-file> [<script-arguments>]

Executes the Dart script <dart-script-file> with the given list of <script-arguments>.

Common VM flags:
--enable-asserts
  Enable assert statements.
--help or -h
  Display this message (add -v or --verbose for information about
  all VM options).
--package-root=<path> or -p<path>
  Where to find packages, that is, "package:..." imports.

[knopp]

Are there any plans of addressing this? Distributing binaries on Mac and Windows that are not codesigned is not feasible (on Windows you will not get through smart screen, on Mac since Catalina you need hardened runtime and notarization, before catalina there's gatekeeper).

[dnfield]

I was hoping to use this to be able to ship a tool from the Flutter engine repo to the Flutter tool. I won't be able to do so without this feature.

[tonosama-atlacatl]

I WANT IT!!!! I Need IT!! I wrote a tool to organize photos/videos/etc... take it from someone that has spent hours helping others install the SDK over the phone... just so they can test my script.

[MichaelCharles]

Is this a planned feature? It seems like there's been relatively silence on this for the past few months.

[alexgoussev]

I very hope this feature will be implemented in the future! I need it!

[stephantual]

Adding a vote for this feature, massively important if desktop dart wants to get into the big leagues.

[ghost]

Windows desktop apps without signning, most of people or security software will not trust them.
Hope fix it quickly.

[jodinathan]

is there a workaround to this?

[Timmmm]

I WANT IT!!!! I Need IT!! I wrote a tool to organize photos/videos/etc... take it from someone that has spent hours helping others install the SDK over the phone... just so they can test my script.

It is still possible to run unsigned binaries on Mac and Windows. On Mac you have to right-click the App and select Open (rather than double clicking it), and on Windows you have to click "More info" and then "Run anyway". Completely not obvious, but you shouldn't need to install the SDK just to get a friend to run something!

Plenty of free software is still distributed unsigned on both platforms, though it would obviously be great if code signing was supported.

[knopp]

I WANT IT!!!! I Need IT!! I wrote a tool to organize photos/videos/etc... take it from someone that has spent hours helping others install the SDK over the phone... just so they can test my script.

It is still possible to run unsigned binaries on Mac and Windows. On Mac you have to right-click the App and select Open (rather than double clicking it), and on Windows you have to click "More info" and then "Run anyway". Completely not obvious, but you shouldn't need to install the SDK just to get a friend to run something!

Plenty of free software is still distributed unsigned on both platforms, though it would obviously be great if code signing was supported.

None of these is feasible. Especially if you want to distribute commercial software to general public.

[Timmmm]

Not for commercial software, no. I just wanted to point out that you don't need to go as far as installing the Dart SDK to work around this!

[etx]

We build a command line app with dart2native the acts as a bridge in Chrome using it's NativeMessaging api. But with the binary in a mac app bundle we can't codesign or notarize 😭

[gkjpettet]

This is a compete dealbreaker for our business. How on Earth can one make a commercial product using dart if you can’t code sign?

Has there been any progress here since the issue has been open for a year?

[thunderstorm010]

Is this being worked on? There is nearly 2 years past when this issue was created, and there is no PR still.

[knopp]

@thunderstorm010, the comment here would sadly suggest otherwise.

[timotheux]

So you can't sign the exe, add icon to it, It is not showing any author Metadata like Author, Version, File version, product name, product version etc., which makes it impossible to identify the origin of the exe.

Infact mraleph stop short of calling applications produced by dart2native a virus by saying "executables produced by dart2native are not exactly adhering to OS standard executable format (PE on Windows, Mach-O on Mac OS X or ELF on Linux)".

How did this advance past the prototyping stage?

[maks]

@timotheux just because it doesn't work for your requirements it doesn't follow that the feature isn't useful to others. Dart exe's are already useful for a number of production use cases, especially serverside, internal tooling etc.

Sure it would make it more useful to be able to be able to sign executables produced by dart2native but please don't imply that it's not fit for purpose already for production use in some environments.

[timotheux]

@maks I am not referring to a use case scenario, a requirement or implying that dart2native is not useful. Lets stay on point here.
I have used dart2native for some project where it performed extremely well in the task but couldn't make it to production because it looks like a suspicious exe.

I started wondering why the Dart team will put in a huge amount of work and leave out the basics. I am simply saying dart2native should not have passed the prototyping stage without an identity. Its like having a car on the road without VIN, Make, Model and paint job.

No country will allow such vehicle on their roads because of the inherent dangers.

[mraleph]

I suggest to take this discussion somewhere else, because it not relevant for this issue. If you have some concrete use case - it is enough to up vote the issue and leave a comment.

We recognise the need to support the signing on platforms like Mac OS X and Windows, but so far we did not have resources to make necessary changes. I am hopeful that eventually we will find an opening and address the issue, but we have not made any concrete decisions on this yet.

@timotheux

I started wondering why the Dart team will put in a huge amount of work and leave out the basics.

That's where you misunderstand how things went - we did an absolute bare minimum of work to support dart2native, because we did not have resources to spare on something more robust and that's why its implementation is so hacky. We considerably underestimated the actual demand for this feature. We expected the usage mostly on Linux in server-side environments, and not deployments of CLI programs on Windows/Mac.

[timotheux]

@mraleph I will let this rest for now, so that my intensions are not misunderstood. I speak frankly because I love dartlang and dart2native, and I don't want it to end up like the Qt framework (which I once loved too).

When it comes to the cross platform frameworks out there, dart2native has almost all the right ideas. Hence, my confusion at the lack of basic functions.

I wish you and your team the best and godspeed as you improve the language and framework we all love to use.

Cheers

[illia-romanenko]

@mraleph, thanks for the insight into your decision-making process and your assumptions about the usage. Also, it's great that you see rising demand in codesigning and the new ways of using dart2native. I'm sure that many people are waiting for this feature, so I hope that managers can prioritize.

In the meantime, could somebody provide a bit more context of that hacky implementation? What is missing, what needs to be done, any relevant documentation (if exists)?

[mraleph]

@illia-romanenko dart2native is currently implemented in the following way: it takes AOT runtime binary (which is a valid platform executable produced by a native C++ toolchain) and an AOT snapshot (ELF file produced by Dart AOT compiler) and simply concatenates them together. From OS perspective the result is still a valid executable file which simply has a bunch of garbage at the end - but OS can still run it. When AOT runtime binary is run it looks at its own file to see if there is a snapshot appended to it - and if there is then it loads and runs it.

Problems begin when you try to edit the binary using tools which work with native executable formats (e.g. sign it, change an icon, etc). For these tools the snapshot at the end of the file are simply garbage bytes, so they usually just remove them.

To fix the problem we need to start properly linking AOT runtime and AOT snapshot together so that the result is a native binary. The simplest way to achieve might be to take a dependency on an existing linker which supports the platforms we support (lld probably fits the bill), figuring out few missing pieces (e.g. we need to figure out what format of object files to feed to lld and implement necessary support in AOT backend, etc), the figuring out packaging (e.g. I don't think lld binary should be shipped with Dart SDK - instead some sort of cloud storage should be setup for on demand SDK components).

[sstrickl]

@Ehesp if you'd like to test the fix, the latest dev channel binary build at https://dart.dev/get-dart/archive should include the fix. If anyone's interested in trying out the Windows side of things, that dev channel build should also include those changes.

Going ahead and marking this as closed since standalone executables for both Windows and Mac OS X should now be signable, but please let us know if anyone runs into any issues :)

[liudonghua123]

Nice work. I can also add an icon to the native executable. see electron/rcedit#91 (comment), #45373.

[dnfield]

@christopherfujino fyi this is what we were discussing yesterday

[mit-mit]

macOS steps:

• compile: dart compile exe bin/[entrypoint].exe
• sign: codesign -s [cert ID] bin/[entrypoint].exe (note: the cert must be in Apple Keychain Access)
• check: codesign -dv --verbose=4 bin/[entrypoint].exe

[jmagman]

macOS steps:

I tested this on flutter_tools on dart 2.17.0-259.0.dev on Xcode 13.3 (beta 3), x64 macOS 12.3.

$ dart compile exe flutter/packages/flutter_tools/bin/flutter_tools.dart
Info: Compiling without sound null safety
Generated:flutter/packages/flutter_tools/bin/flutter_tools.exe
$ codesign -s [cert ID] flutter/packages/flutter_tools/bin/flutter_tools.exe
$ codesign -dv --verbose=4 flutter/packages/flutter_tools/bin/flutter_tools.exe
Executable=/path/to/flutter/packages/flutter_tools/bin/flutter_tools.exe
Identifier=flutter_tools
Format=Mach-O thin (x86_64)
CodeDirectory v=20400 size=184305 flags=0x0(none) hashes=5754+2 location=embedded
VersionPlatform=1
VersionMin=658688
VersionSDK=721152
Hash type=sha256 size=32
CandidateCDHash sha256=7ca0370ce25ce274ecab0798ce26d1c5aa75dd69
CandidateCDHashFull sha256=7ca0370ce25ce274ecab0798ce26d1c5aa75dd69cb0e7ee8b39b975229745d9c
Hash choices=sha256
CMSDigest=7ca0370ce25ce274ecab0798ce26d1c5aa75dd69cb0e7ee8b39b975229745d9c
CMSDigestType=2
Executable Segment base=0
Executable Segment limit=3784704
Executable Segment flags=0x1
Page size=4096
CDHash=7ca0370ce25ce274ecab0798ce26d1c5aa75dd69
Signature size=4782
Authority=Apple Development: [Correct cert name]
Authority=Apple Worldwide Developer Relations Certification Authority
Authority=Apple Root CA
Signed Time=Mar 31, 2022 at 11:12:59 AM
Info.plist=not bound
TeamIdentifier=[Correct Team ID]
Sealed Resources=none
Internal requirements count=1 size=176

Signature is valid and satisfies its Designated Requirement.
And it runs:

$ flutter/packages/flutter_tools/bin/flutter_tools.exe -h
Manage your Flutter app development.

Common commands
...

Note: [cert ID] can be found via security find-identity -p codesigning.

[khainke]

@Ehesp if you'd like to test the fix, the latest dev channel binary build at https://dart.dev/get-dart/archive should include the fix. If anyone's interested in trying out the Windows side of things, that dev channel build should also include those changes.

Going ahead and marking this as closed since standalone executables for both Windows and Mac OS X should now be signable, but please let us know if anyone runs into any issues :)

Is this fix already released / on the stable channel?

[mit-mit]

It is not (but we're getting close). You can try it using 2.17 from the beta channel:
https://dart.dev/get-dart/archive#beta-channel

[khainke]

I have problems signing dart executables with codesign for macOS with enabled hardened runtime ...

>> dart compile exe -o bin/test bin/test.dart
Info: Compiling with sound null safety
Generated: /Users/user/projects/dart_playground/bin/test
>> ./bin/test
Hello World
>> codesign --sign "Cert" --identifier com.example.test -o runtime -f --timestamp bin/test
bin/test: replacing existing signature
>> ./bin/test
[1]    79487 killed     ./bin/test

dart code (bin/test.dart):

main(){
  print('Hello World');
}

[mraleph]

@khainke You probably need to preserve entitlements (try adding --preserve-metadata=entitlements to codesign).

[khainke]

That does not work for me, but adding the com.apple.security.cs.allow-unsigned-executable-memoryworks. But why is this necessary?

Release.entitlements:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
</dict>
</plist>

>> dart compile exe -o bin/test bin/test.dart
Info: Compiling with sound null safety
Generated: /Users/user/projects/dart_playground/bin/test
>> codesign --sign "Cert" --identifier com.example.test -o runtime --entitlements lib/macos/Release.entitlements --timestamp bin/test
>> ./bin/test
Hello World

[mraleph]

But why is this necessary?

Because snapshot itself is still not represented as a native format. I have not reviewed the implementation but I guess we still just put an ELF file into a Mach-O file (though respecting Mach-O structure now) and then use our ELF loader to load it up.

To drop allow-unsigned-executable-memory we need to put snapshots executable code into the text segment.

[techouse]

Just tested it with Dart 2.17.0 and it works like a charm :)

#!/usr/bin/env bash

dart compile exe bin/main.dart -o build/app
codesign -v -s XXXXXXXXXX -i com.example.test build/app
codesign -dv --verbose=4 build/app

[johnpryan]

For more information about signing, see the signing section on the dart compile page.

[techouse]

I can confirm that I've successfully signed and notarised a compiled binary with Apple following this exhaustive StackOverflow answer. 🚀

Because I spent hours researching this not so straightforward process, I wrote a small script to help me do so.

#!/usr/bin/env bash

# This script should be a starting point to help you sign and notarise a Mach-O binary app.
# Read it carefully and replace any placeholders with actual data.

# Sign the compiled binary
codesign \
  --sign="XXXX" \                        # replace with hash of "Developer ID Application: Your name (Your Team)"
  --identifier="com.example.test" \      # replace with app's bundle id
  --deep \
  --force \
  --options=runtime \
  --entitlement="./entitlements.plist" \ # must allow com.apple.security.cs.allow-unsigned-executable-memory
  --timestamp \
  --verbose=4 \
  ./path/to/compiled/exe

# Verify the signed binary
codesign -dv --verbose=4 ./path/to/compiled/exe

# ZIP the binary because altool won't accept a raw Mach-O binary
zip -j ./path/to/compiled/exe.zip ./path/to/compiled/exe

# Notarize the binary in ZIP form
xcrun altool \
  --notarize-app \
  --primary-bundle-id="com.example.test" \  # replace with app's bundle id
  --username="[email protected]" \   # Apple ID username 
  --password="@keychain:Developer-altool" \ # create an app-specific password; https://support.apple.com/en-us/HT204397
  --asc-provider="XXXXXX" \                 # your team
  --file="./path/to/compiled/exe.zip"

# Delete zip file as it's no longer needed at this point
# Apple will notarize the Mach-O binary inside the ZIP
rm ./path/to/compiled/exe.zip

# Wait a while then verify your Mach-O binary
spctl -a -vvv -t install ./path/to/compiled/exe

# In case you encountered an error run and check the output
xcrun altool \
  --notarization-info "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" \  # the UUID altool gave you
  --username "[email protected]" \                       # Apple ID username
  --password "@keychain:Developer-altool"                       # same app-specific password

[ArtGangsta]

It's a bit silly to have implemented the feature in this way, tagging the binary at the end of created EXE. That's something we did back in the 90s.
It was clearly done out of ignorance and convenience. The proper solution was to include an alternative which requires native toolking support, and simply "sucks in" the binary into the executable in some form. I mean, you can literally link in any arbitrary file into an executable, and it's available as an "extern" from C. You can determine what it is using objdump -t (from gcc).
Rather than delivered the "exe" that you concact with, you would have delivered the object files for that platform, and require the developer to have at least a linker available -- or maybe figure out a way to include a redistributable compatible linker with the SDK.

Honestly for something as ambitious as dart2native, the incremental work would have been trivial compared to the overall effort, and certainly would have completed the story. Let's be honest -- it wasn't considered until the alternative approach was working, and only then we ran out of time.

[Rexios80]

I'll make this super easy for future people that end up here:

1. Get a "Developer ID Application" certificate
   a. Open Xcode
   b. Go to Preferences > Accounts > Manage Certificates
   c. Click the "+" button and create one
2. Install gon
   a. brew install mitchellh/gon/gon
3. Create a gon.json file

{
    "source": [
        "{your_app_exe}"
    ],
    "bundle_id": "{com.example.yourDartApp}",
    "apple_id": {
        "username": "{your email}",
        "password": "@env:NOTARIZATION_PASSWORD",
        "provider": "{your App Store Connect Team ID}"
    },
    "sign": {
        "application_identity": "Developer ID Application: {your team name}",
        "entitlements_file": "Release.entitlements"
    },
    "zip": {
        "output_path": "build.zip"
    }
}

4. Create a Release.entitlements file

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
</dict>
</plist>

5. Go to https://appleid.apple.com/account/manage and create an "App-Specific Password"
6. Run in terminal: export NOTARIZATION_PASSWORD={your_app_specific_password}
7. Run in terminal: gon gon.json

And that should be it

[Rexios80]

Okay apparently that doesn't work even though all the tools are saying the thing is signed properly...

I tried doing exactly what @techouse specified and that has the same result

[techouse]

@Rexios80 I've used that technique of mine a few days ago and it works like a charm.

Check your certificates (you are probably aware of the fact that you need a paid Apple Developer license) and entitlements.

[Rexios80]

Do I need any entitlements besides com.apple.security.cs.allow-unsigned-executable-memory? Before I added that my app just instantly crashed even after bypassing the unidentified developer warnings. Now my app runs fine, but macOS still claims it's unidentified. I would imagine notarization would fail if my certificates were invalid.

[techouse]

@Rexios80 did you get an email from Apple saying that your binary has been successfully notarized?

[Rexios80]

@techouse Yes that's why this is so weird

[Rexios80]

Also the notarization logs show no issues

[cbenhagen]

To drop allow-unsigned-executable-memory we need to put snapshots executable code into the text segment.

@mraleph is there already an open issue for this?