Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Build Web SDK in wasm build tree #36520

Merged
merged 19 commits into from
Oct 14, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
543b02d
Build web sdk in wasm tree.
eyebrowsoffire Oct 7, 2022
07690ee
Use prebuilts to build tests.
eyebrowsoffire Oct 8, 2022
2259916
Don't create the copy target if `flutter_prebuilt_dart_sdk` is false
eyebrowsoffire Oct 8, 2022
d1a9f0a
Felt test without build.
eyebrowsoffire Oct 8, 2022
4650b9e
Remove debug statement I accidentally left in.
eyebrowsoffire Oct 8, 2022
1041f93
Refactored ddc and outline generation in gn files.
eyebrowsoffire Oct 10, 2022
e4f0c0d
Remove dependency on dart-sdk copy.
eyebrowsoffire Oct 10, 2022
81eff9c
Formatting stuff.
eyebrowsoffire Oct 10, 2022
82812b6
Use host_prebuilt_dart_sdk instead of just prebuilt_dart_sdk
eyebrowsoffire Oct 10, 2022
f3b6cfd
Don't make the flutter_web_sdk artifact on android builds, is isn't u…
eyebrowsoffire Oct 10, 2022
a9e7f3f
Change strategy for short-circuiting the flutter_web_sdk archive.
eyebrowsoffire Oct 11, 2022
9d1d6ae
Merge branch 'main' into web_sdk_build
eyebrowsoffire Oct 11, 2022
8bfb22f
Don't produce a pointless xcode/VS project when building wasm.
eyebrowsoffire Oct 11, 2022
094c678
Change up the way we are treating flags, and package canvaskit with t…
eyebrowsoffire Oct 13, 2022
d81efbd
Force `--build-canvaskit` until we update the recipes.
eyebrowsoffire Oct 13, 2022
a0d5345
Merge branch 'main' into web_sdk_build
eyebrowsoffire Oct 13, 2022
aeaa119
Oops, only build canvaskit in the web case.
eyebrowsoffire Oct 14, 2022
3d667d2
flutter_web_sdk need not be built on in linux_host_release.
eyebrowsoffire Oct 14, 2022
924697d
Change target for linux_host_engine drone.
eyebrowsoffire Oct 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 3 additions & 25 deletions BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ import("//third_party/dart/build/dart/copy_tree.gni")

# Whether to build the dartdevc sdk, libraries, and source files
# required for the flutter web sdk.
# TODO(jacksongardner): remove this poorly named argument once the recipes stop
# using it.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please link to a github issue for all TODOs.

declare_args() {
full_dart_sdk = false
}
Expand Down Expand Up @@ -44,30 +46,6 @@ config("export_dynamic_symbols") {
}
}

if (flutter_prebuilt_dart_sdk) {
copy_trees("_copy_trees") {
sources = [
{
target = "copy_dart_sdk"
visibility = [ ":dart_sdk" ]
source = prebuilt_dart_sdk
dest = "$root_out_dir/dart-sdk"
ignore_patterns = "{}"
},
]
}
}

group("dart_sdk") {
if (build_engine_artifacts) {
if (flutter_prebuilt_dart_sdk) {
public_deps = [ ":copy_dart_sdk" ]
} else {
public_deps = [ "//third_party/dart:create_sdk" ]
}
}
}

group("flutter") {
testonly = true

Expand Down Expand Up @@ -96,7 +74,7 @@ group("flutter") {

if (build_engine_artifacts) {
public_deps += [
":dart_sdk",
"//flutter/dart_sdk",
"//flutter/flutter_frontend_server:frontend_server",

# This must be listed explicitly for desktop cross-builds since
Expand Down
7 changes: 0 additions & 7 deletions build/archives/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -220,13 +220,6 @@ if (build_engine_artifacts && !flutter_prebuilt_dart_sdk) {
}
}

# Archives Flutter Web SDK
if (!is_fuchsia) {
group("flutter_web_sdk") {
deps = [ "//flutter/web_sdk:flutter_web_sdk_archive" ]
}
}

# Archives Flutter Windows Artifacts
if (host_os == "win") {
zip_bundle("windows_flutter") {
Expand Down
5 changes: 2 additions & 3 deletions ci/builders/linux_host_engine.json
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
"flutter/build/archives:embedder",
"flutter/build/archives:flutter_patched_sdk",
"flutter/build/archives:dart_sdk_archive",
"flutter/build/archives:flutter_web_sdk",
"flutter/web_sdk",
"flutter/tools/font-subset",
"flutter/shell/platform/linux:flutter_gtk"
]
Expand Down Expand Up @@ -151,8 +151,7 @@
"targets": [
"flutter:unittests",
"flutter/build/archives:flutter_patched_sdk",
"flutter/shell/platform/linux:flutter_gtk",
"flutter/build/archives:flutter_web_sdk"
"flutter/shell/platform/linux:flutter_gtk"
]
},
"tests": [
Expand Down
4 changes: 3 additions & 1 deletion common/config.gni
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ declare_args() {

# Whether to use a prebuilt Dart SDK instead of building one.
flutter_prebuilt_dart_sdk = false

build_flutter_web_sdk = false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment for this arg.

}

# feature_defines_list ---------------------------------------------------------
Expand Down Expand Up @@ -114,7 +116,7 @@ if (flutter_prebuilt_dart_sdk) {
# There is no prebuilt Dart SDK targeting Fuchsia, but we also don't need
# one, so even when the build is targeting Fuchsia, use the prebuilt
# Dart SDK for the host.
if (current_toolchain == host_toolchain || is_fuchsia) {
if (current_toolchain == host_toolchain || is_fuchsia || is_wasm) {
prebuilt_dart_sdk = host_prebuilt_dart_sdk
prebuilt_dart_sdk_config = _host_prebuilt_dart_sdk_config
} else {
Expand Down
24 changes: 24 additions & 0 deletions dart_sdk/BUILD.gn
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import("//flutter/common/config.gni")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file needs a copyright header

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two things:

  1. What is the reason for moving this?
  2. GN files need a copyright header.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original reason for moving this was that the flutter/BUILD.gn file is not wasm friendly (it transitively pulls in a bunch of .gni files that have platform checks that choke when the target os/arch is wasm). Since I was trying to build the web sdk in the wasm build, it was pulling in the flutter/BUILD.gn file when it had a dependency on the dart-sdk target, so I separated it out and simplified it so that we wouldn't have to pull in the whole of flutter/BUILD.gn when building the web sdk.

After some additional changes though, the web sdk build now uses the prebuilt sdk in-place and so does not have a dependency on the dart_sdk target anymore, so I can actually move it back if this bothers you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to allow Dart SDK hackers to test Dart SDK builds with local source changes, so we may end up with this dependency again. If it's better to have it split out into a standalone BUILD.gn I think I'd rather keep it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to allow Dart SDK hackers to test Dart SDK builds with local source changes, so we may end up with this dependency again. If it's better to have it split out into a standalone BUILD.gn I think I'd rather keep it.

I don't follow how that's related. The existing way to hack on the Dart SDK in the engine tree is to make changes under third_party/dart, and to use the --no-prebuilt-dart-sdk flag to the gn script. Does that not work for some reason?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the dart-sdk target is in flutter/BUILD.gn, then the wasm build cannot consume or use it because the wasm build can't actually touch flutter/BUILD.gn without pulling in transitive dependencies that break under a wasm build. The transitive dependencies break even if you aren't building any of their targets, and they are dependencies we don't necessarily have control over.

That being said, we aren't actually using the dart-sdk target for the web build anymore, we just use prebuilts. I also don't actually see us adding a dependency on the dart-sdk target in the future either, and if we did it would be in a different manner that would probably build it with the non-default toolchain, since it's weird to pull host artifacts into the wasm build tree. So actually for our purposes I don't care anymore, and if you want me to refactor that target back into flutter/BUILD.gn I don't have any real practical objection, but I don't actually understand why that would be desirable. flutter/BUILD.gn is a bit of a kitchen sink and probably would be easier to work with if it were broken up into smaller logical components, and so having the dart-sdk separated into a different file seems better to me than keeping it in flutter/BUILD.gn. But again we don't really depend on either any more, so I don't mind either way you want to go with it.

import("//third_party/dart/build/dart/copy_tree.gni")

if (flutter_prebuilt_dart_sdk) {
copy_trees("_copy_trees") {
sources = [
{
target = "copy_dart_sdk"
visibility = [ ":dart_sdk" ]
source = prebuilt_dart_sdk
dest = "$root_out_dir/dart-sdk"
ignore_patterns = "{}"
},
]
}
}

group("dart_sdk") {
if (flutter_prebuilt_dart_sdk) {
public_deps = [ ":copy_dart_sdk" ]
} else {
public_deps = [ "//third_party/dart:create_sdk" ]
}
}
46 changes: 13 additions & 33 deletions lib/web_ui/dev/build.dart
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
// found in the LICENSE file.

import 'dart:async';
import 'dart:io' show Directory, Platform;
import 'dart:io' show Directory;

import 'package:args/command_runner.dart';
import 'package:path/path.dart' as path;
Expand All @@ -25,6 +25,7 @@ class BuildCommand extends Command<bool> with ArgUtils<bool> {
'build-canvaskit',
help: 'Build CanvasKit locally instead of getting it from CIPD. Disabled '
'by default.',
defaultsTo: true
);
}

Expand All @@ -42,15 +43,9 @@ class BuildCommand extends Command<bool> with ArgUtils<bool> {
FutureOr<bool> run() async {
final FilePath libPath = FilePath.fromWebUi('lib');
final List<PipelineStep> steps = <PipelineStep>[
GnPipelineStep(),
NinjaPipelineStep(target: environment.engineBuildDir),
GnPipelineStep(buildCanvasKit: buildCanvasKit),
NinjaPipelineStep(target: environment.wasmReleaseOutDir),
];
if (buildCanvasKit) {
steps.addAll(<PipelineStep>[
GnPipelineStep(target: 'canvaskit'),
NinjaPipelineStep(target: environment.wasmReleaseOutDir),
]);
}
final Pipeline buildPipeline = Pipeline(steps: steps);
await buildPipeline.run();

Expand All @@ -73,39 +68,24 @@ class BuildCommand extends Command<bool> with ArgUtils<bool> {
/// Not safe to interrupt as it may leave the `out/` directory in a corrupted
/// state. GN is pretty quick though, so it's OK to not support interruption.
class GnPipelineStep extends ProcessStep {
GnPipelineStep({this.target = 'engine'})
: assert(target == 'engine' || target == 'canvaskit');
GnPipelineStep({required this.buildCanvasKit});

final bool buildCanvasKit;

@override
String get description => 'gn';

@override
bool get isSafeToInterrupt => false;

/// The target to build with gn.
///
/// Acceptable values: engine, canvaskit
final String target;

@override
Future<ProcessManager> createProcess() {
print('Running gn for $target...');
final List<String> gnArgs = <String>[];
if (target == 'engine') {
gnArgs.addAll(<String>[
'--unopt',
if (Platform.isMacOS) '--xcode-symlinks',
'--full-dart-sdk',
if (environment.isMacosArm) '--mac-cpu=arm64',
]);
} else if (target == 'canvaskit') {
gnArgs.addAll(<String>[
'--wasm',
'--runtime-mode=release',
]);
} else {
throw StateError('Target was not engine or canvaskit: $target');
}
print('Running gn...');
final List<String> gnArgs = <String>[
'--web',
'--runtime-mode=release',
if (buildCanvasKit) '--build-canvaskit',
];
return startProcess(
path.join(environment.flutterDirectory.path, 'tools', 'gn'),
gnArgs,
Expand Down
15 changes: 3 additions & 12 deletions lib/web_ui/dev/environment.dart
Original file line number Diff line number Diff line change
Expand Up @@ -20,21 +20,17 @@ Environment? _environment;
class Environment {
factory Environment() {
final bool isMacosArm = ffi.Abi.current() == ffi.Abi.macosArm64;
final io.File dartExecutable = io.File(io.Platform.resolvedExecutable);
final io.File self = io.File.fromUri(io.Platform.script);

final io.Directory engineSrcDir = self.parent.parent.parent.parent.parent;
final io.Directory engineToolsDir =
io.Directory(pathlib.join(engineSrcDir.path, 'flutter', 'tools'));
final io.Directory outDir =
io.Directory(pathlib.join(engineSrcDir.path, 'out'));
final io.Directory engineBuildDir =
io.Directory(pathlib.join(
outDir.path,
isMacosArm ? 'host_debug_unopt_arm64' : 'host_debug_unopt'
));
final io.Directory wasmReleaseOutDir =
io.Directory(pathlib.join(outDir.path, 'wasm_release'));
final io.Directory dartSdkDir =
io.Directory(pathlib.join(engineBuildDir.path, 'dart-sdk'));
final io.Directory dartSdkDir = dartExecutable.parent.parent;
final io.Directory webUiRootDir = io.Directory(
pathlib.join(engineSrcDir.path, 'flutter', 'lib', 'web_ui'));

Expand All @@ -55,7 +51,6 @@ class Environment {
engineSrcDir: engineSrcDir,
engineToolsDir: engineToolsDir,
outDir: outDir,
engineBuildDir: engineBuildDir,
wasmReleaseOutDir: wasmReleaseOutDir,
dartSdkDir: dartSdkDir,
);
Expand All @@ -68,7 +63,6 @@ class Environment {
required this.engineSrcDir,
required this.engineToolsDir,
required this.outDir,
required this.engineBuildDir,
required this.wasmReleaseOutDir,
required this.dartSdkDir,
});
Expand All @@ -93,9 +87,6 @@ class Environment {
/// This is where you'll find the ninja output, such as the Dart SDK.
final io.Directory outDir;

/// The output directory for the host_debug_unopt build.
final io.Directory engineBuildDir;

/// The output directory for the wasm_release build.
///
/// We build CanvasKit in release mode to reduce code size.
Expand Down
59 changes: 47 additions & 12 deletions tools/gn
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ SRC_ROOT = os.path.dirname(
def get_out_dir(args):
if args.target_os is not None:
target_dir = [args.target_os]
elif args.web:
target_dir = ['wasm']
else:
target_dir = ['host']

Expand Down Expand Up @@ -251,10 +253,15 @@ def to_gn_args(args):

# If building for WASM, set the GN args using 'to_gn_wasm_args' as most
# of the Flutter SDK specific arguments are unused.
if args.target_os == 'wasm':
if args.target_os == 'wasm' or args.web:
to_gn_wasm_args(args, gn_args)
return gn_args

gn_args['build_flutter_web_sdk'] = args.build_web_sdk
gn_args['full_dart_sdk'] = args.full_dart_sdk
if args.build_web_sdk or args.full_dart_sdk:
gn_args['build_canvaskit'] = args.build_canvaskit

if args.enable_unittests:
gn_args['enable_unittests'] = args.enable_unittests

Expand Down Expand Up @@ -359,7 +366,6 @@ def to_gn_args(args):
gn_args['embedder_for_target'] = args.embedder_for_target
gn_args['dart_lib_export_symbols'] = False
gn_args['flutter_runtime_mode'] = runtime_mode
gn_args['full_dart_sdk'] = args.full_dart_sdk
gn_args['dart_version_git_info'] = not args.no_dart_version_git_info

gn_args['dart_lib_export_symbols'] = False
Expand Down Expand Up @@ -627,6 +633,12 @@ def to_gn_wasm_args(args, gn_args):
gn_args['skia_canvaskit_enable_webgpu'] = False
is_profile_build = args.runtime_mode == 'profile' or args.runtime_mode == 'debug'
gn_args['skia_canvaskit_profile_build'] = is_profile_build
gn_args['flutter_prebuilt_dart_sdk'] = True

# TODO(jacksongardner): Make this based off of the input argument rather
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto comment about linking TODOs to github issues.

# than forced to true, once the recipes are updated.
gn_args['build_canvaskit'] = True
gn_args['build_flutter_web_sdk'] = True


def parse_args(args):
Expand Down Expand Up @@ -702,6 +714,7 @@ def parse_args(args):
parser.add_argument(
'--wasm', dest='target_os', action='store_const', const='wasm'
)
parser.add_argument('--web', action='store_true', default=False)
parser.add_argument(
'--windows', dest='target_os', action='store_const', const='win'
)
Expand Down Expand Up @@ -825,6 +838,27 @@ def parse_args(args):
'--no-full-dart-sdk', dest='full_dart_sdk', action='store_false'
)

parser.add_argument(
'--build-web-sdk',
default=False,
action='store_true',
help='build the flutter web sdk'
)
parser.add_argument(
'--no-build-web-sdk', dest='build_web_sdk', action='store_false'
)

parser.add_argument(
'--build-canvaskit',
default=False,
action='store_true',
help='build canvaskit from source'
)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For other args there's no extra newline between the arg and the --no version.

parser.add_argument(
'--no-build-canvaskit', dest='build_canvaskit', action='store_false'
)

parser.add_argument(
'--ide',
default='',
Expand Down Expand Up @@ -968,16 +1002,17 @@ def main(argv):
'--export-compile-commands',
]

if args.ide != '':
command.append('--ide=%s' % args.ide)
elif sys.platform == 'darwin':
# On the Mac, generate an Xcode project by default.
command.append('--ide=xcode')
command.append('--xcode-project=flutter_engine')
command.append('--xcode-build-system=new')
elif sys.platform.startswith('win'):
# On Windows, generate a Visual Studio project.
command.append('--ide=vs')
if args.target_os != 'wasm':
if args.ide != '':
command.append('--ide=%s' % args.ide)
elif sys.platform == 'darwin':
# On the Mac, generate an Xcode project by default.
command.append('--ide=xcode')
command.append('--xcode-project=flutter_engine')
command.append('--xcode-build-system=new')
elif sys.platform.startswith('win'):
# On Windows, generate a Visual Studio project.
command.append('--ide=vs')

command.append('--export-compile-commands=default')

Expand Down
6 changes: 5 additions & 1 deletion wasm/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@
# //flutter/BUILD.gn pulls in Flutter SDK dependencies which will crash
# when the target CPU is WASM.

import("//flutter/common/config.gni")

# This is the default target when building when the target CPU is WASM.
group("wasm") {
deps = [ "//third_party/skia/modules/canvaskit" ]
if (build_flutter_web_sdk) {
deps = [ "//flutter/web_sdk" ]
}
}
Loading