Cannot correctly build before COPY on release mode for macos

[australiaelon]

when run with —debug, everything works

flutter build macos --debug
--- xcodebuild: WARNING: Using the first of multiple matching destinations:
{ platform:macOS, arch:arm64, id:xxx, name:My Mac }
{ platform:macOS, arch:x86_64, id:xxx, name:My Mac }
Building macOS application…

However, when run with —release, it failed to copy, I guess it is due to it spend more time to create framework on release, thus too small wait time before coping thus causing this issue.

flutter build macos --release
--- xcodebuild: WARNING: Using the first of multiple matching destinations:
{ platform:macOS, arch:arm64, id:xxxxxxxxxx, name:My Mac }
{ platform:macOS, arch:x86_64, id:xxxxxxxxxx, name:My Mac }
Unhandled exception:
PathNotFoundException: Cannot copy file to '~/app/.dart_tool/native_assets_builder/xxxx/a1fa1bd08d8b815c164ed6bedeadb980/out/libxxxx.dylib', path = '~/xxxxCLI/native/macos/x64/libxxxx.dylib' (OS Error: No such file or directory, errno = 2)
#0      _checkForErrorResponse (dart:io/common.dart:58:9)
#1      _File.copy.<anonymous closure> (dart:io/file_impl.dart:409:7)
<asynchronous suspension>
#2      main.<anonymous closure> (file://~/xxxxCLI/hook/build.dart:29:5)
<asynchronous suspension>
#3      build (package:native_assets_cli/src/api/build.dart:107:3)
<asynchronous suspension>
#4      main (file://~/xxxxCLI/hook/build.dart:17:3)
<asynchronous suspension>

  Building assets for package:xxxx failed.
  build.dart returned with exit code: 255.
  To reproduce run:
  (cd ~/xxxxCLI/; /opt/flutter/bin/cache/dart-sdk/bin/dart --packages=~/app/.dart_tool/package_config.json ~/app/.dart_tool/native_assets_builder/xxxx/a1fa1bd08d8b815c164ed6bedeadb980/hook.dill --config=~/app/.dart_tool/native_assets_builder/xxxx/a1fa1bd08d8b815c164ed6bedeadb980/input.json )
  stderr:
  Unhandled exception:
PathNotFoundException: Cannot copy file to '~/app/.dart_tool/native_assets_builder/xxxx/a1fa1bd08d8b815c164ed6bedeadb980/out/libxxxx.dylib', path = '~/xxxxCLI/native/macos/x64/libxxxx.dylib' (OS Error: No such file or directory, errno = 2)
#0      _checkForErrorResponse (dart:io/common.dart:58:9)
#1      _File.copy.<anonymous closure> (dart:io/file_impl.dart:409:7)
<asynchronous suspension>
#2      main.<anonymous closure> (file://~/xxxxCLI/hook/build.dart:29:5)
<asynchronous suspension>
#3      build (package:native_assets_cli/src/api/build.dart:107:3)
<asynchronous suspension>
#4      main (file://~/xxxxCLI/hook/build.dart:17:3)
<asynchronous suspension>

  stdout:


Building App.framework for x86_64...
Building App.framework for arm64...
Target dart_build failed: Error: Building native assets failed. See the logs for more details.

Command PhaseScriptExecution failed with a nonzero exit code
** BUILD FAILED **

And for the lib, we have

import 'dart:io';
import 'package:native_assets_cli/code_assets.dart';

LinkMode getLinkMode(LinkModePreference preference) {
  if (preference == LinkModePreference.dynamic ||
      preference == LinkModePreference.preferDynamic) {
    return DynamicLoadingBundled();
  }
  assert(
    preference == LinkModePreference.static ||
        preference == LinkModePreference.preferStatic,
  );
  return StaticLinking();
}

void main(List<String> args) async {
  await build(args, (input, output) async {
    final os = input.config.code.targetOS;
    final arch = input.config.code.targetArchitecture;
    final linkMode = getLinkMode(input.config.code.linkModePreference);

    final libraryFileName = os.libraryFileName(input.packageName, linkMode);
    final libUri = input.outputDirectory.resolve(libraryFileName);

    final prebuiltPath = 'native/$os/$arch/$libraryFileName';
    final prebuiltLibUri = input.packageRoot.resolve(prebuiltPath);

    await Directory.fromUri(input.outputDirectory).create(recursive: true);
    await File.fromUri(prebuiltLibUri).copy(libUri.toFilePath());

    print('$libraryFileName');

    output.assets.code.add(
      CodeAsset(
        package: input.packageName,
        name: 'ffi_bindings.dart',
        file: libUri,
        linkMode: linkMode,
        os: input.config.code.targetOS,
        architecture: input.config.code.targetArchitecture,
      ),
    );
  });
}

with config file,

environment:
  sdk: ">=3.7.2 <4.0.0"

dependencies:
  ffi: ^2.1.4
  native_assets_cli: ^0.12.0

dev_dependencies:
  lints: ^5.0.0
  test: ^1.25.8

for the app, we have config file,

environment:
  sdk: ">=3.7.2 <4.0.0"
  flutter: 3.29.2

dependencies:
  flutter:
    sdk: flutter
  cupertino_icons: ^1.0.8
  shared_preferences: ^2.5.3
  json_annotation: ^4.9.0
  path_provider: ^2.1.5
  fl_chart: ^0.70.2
  provider: ^6.1.4
  logger: ^2.5.0
  intl: ^0.20.2
  xxxlib:
    path: ..

dev_dependencies:
  flutter_test:
    sdk: flutter
  flutter_lints: ^5.0.0

flutter:
  uses-material-design: true

[dcharkes]

Hi @australiaelon!

Thanks for the report.

Let me understand your use case.

However, when run with —release, it failed to copy, I guess it is due to it spend more time to create framework on release, thus too small wait time before coping thus causing this issue.

What do you mean with wait time?

The hook/build.dart should not rely on anything happening outside it. Is native/$os/$arch/$libraryFileName already on your file system before you start the flutter build?

[australiaelon]

yes, we prebuilt all assets to directly implement the ability of native_assets_cli: ^0.12.0 without build.

It works when build on debug mode, but not work on release mode of macos.

BTW, the IOS not working when dealing with static pre-built assets (.a) due to input.config.code.linkModePreference not match, however, we dont know where to change it. And apple dont allow to build the dynamic lib for IOS for security reasons.

[dcharkes]

yes, we prebuilt all assets to directly implement the ability of native_assets_cli: ^0.12.0 without build.

It works when build on debug mode, but not work on release mode of macos.

Could you elaborate why it doesn't work in release mode?

flutter create --template=package_ffi xxxx works fine in both debug and release mode. So what is different about your setup?

What do you mean with wait time? Why is it different in debug and release? Why do you need a wait time?

BTW, the IOS not working when dealing with static pre-built assets (.a) due to input.config.code.linkModePreference not match, however, we don't know where to change it.

Static linking is not yet supported.

(It is not really defined what one would statically link against. E.g. on Android the main application is a JAR and run by the JVM, so there is no native executable to link against. On the other platforms the main application is the Dart VM, the Dart program is a dynamic library. Since the calls to native code will be in the Dart program, it would make more sense to statically link the Dart program and the native code, but this would still result in a dynamic library. For some more info see dart-lang/sdk#47718.)

And apple doesn't allow to build the dynamic lib for IOS for security reasons.

This is not true. As long as dynamic libraries as packaged in a framework with the right metadata using dynamic libaries is fine. If you output a dynamic library from hook/build.dart it will be automatically packaged by Flutter in a framework.

[australiaelon]

This is not true. As long as dynamic libraries as packaged in a framework with the right metadata using dynamic libaries is fine. If you output a dynamic library from hook/build.dart it will be automatically packaged by Flutter in a framework.

Okay, I fix it by add ios flag. It now can build dynamic lib for iOS platform

flutter create --template=package_ffi xxxx works fine in both debug and release mode. So what is different about your setup?

yes, it works fine

[australiaelon]

the main modifications is I replaced hook/build.dart with

import 'dart:io';
import 'package:native_assets_cli/code_assets.dart';

LinkMode getLinkMode(LinkModePreference preference) {
  if (preference == LinkModePreference.dynamic ||
      preference == LinkModePreference.preferDynamic) {
    return DynamicLoadingBundled();
  }
  assert(
    preference == LinkModePreference.static ||
        preference == LinkModePreference.preferStatic,
  );
  return StaticLinking();
}

void main(List<String> args) async {
  await build(args, (input, output) async {
    final os = input.config.code.targetOS;
    final arch = input.config.code.targetArchitecture;
    final linkMode = getLinkMode(input.config.code.linkModePreference);

    final libraryFileName = os.libraryFileName(input.packageName, linkMode);
    final libUri = input.outputDirectory.resolve(libraryFileName);

    final prebuiltPath = 'native/$os/$arch/$libraryFileName';
    final prebuiltLibUri = input.packageRoot.resolve(prebuiltPath);

    await Directory.fromUri(input.outputDirectory).create(recursive: true);
    await File.fromUri(prebuiltLibUri).copy(libUri.toFilePath());

    print('$libraryFileName');

    output.assets.code.add(
      CodeAsset(
        package: input.packageName,
        name: 'ffi_bindings.dart',
        file: libUri,
        linkMode: linkMode,
        os: input.config.code.targetOS,
        architecture: input.config.code.targetArchitecture,
      ),
    );
  });
}```

And place the pre-build lib in  'native/$os/$arch/$libraryFileName'

[australiaelon]

For more clear state, I just crafted a example repo which demonstrates the details for the modification after flutter create --template=package_ffi example

And it seems that simple copy the pre-built lib from src to dst not work for apple platforms, where mentioned on,

    final libraryFileName = os.libraryFileName(input.packageName, linkMode);
    final libUri = input.outputDirectory.resolve(libraryFileName);

    final prebuiltPath = 'native/$os/$arch/$libraryFileName';
    final prebuiltLibUri = input.packageRoot.resolve(prebuiltPath);

    await Directory.fromUri(input.outputDirectory).create(recursive: true);
    await File.fromUri(prebuiltLibUri).copy(libUri.toFilePath());

[australiaelon]

okay, I found out that the path final arch = input.config.code.targetArchitecture; is not standard, and I composed a transfer function, it works~