[wasm/wasi] Consolidate build targets #95775

radical · 2023-12-08T01:18:35Z

[wasm/wasi] Consolidate build targets

Current state of build files:

    wasm: WasmApp.props, WasmApp.targets, WasmApp.Native.targets
    wasi: WasiApp.props, WasiApp.targets, WasiApp.Native.targets

The wasm, and wasi build have lot of shared code but that is not
representative in the actual files, since the wasi targets started life
as a quick-copy-comment-out-bits of the wasm targets.

This commit consolidates these into:

    common: WasmApp.Common.props, WasmApp.Common.targets
    wasm : WasmApp.props, WasmApp.targets
    wasi : WasiApp.props, WasiApp.targets

`WasmApp.Common.{props,targets}`

This has all the common parts of the build for browser-wasm, and wasi,
and includes bits from WasmApp.{props,targets}, and
WasmApp.Native.{props,targets}.

The top level target remains the same - WasmBuildApp.
There are a few "public" targets that can be hooked into:
- PrepareInputsForWasmBuild
- WasmGenerateAppBundle
- PrepareForWasmBuildNative
- WasmLinkDotNet
all these public targets have corresponding *DependsOn properties
which can be used for extending the build

note: this commit does not add a public target for AOT, but it might be
added in future.

`WasmApp.{props,targets}`

This is for browser-wasm projects. The file might be renamed in
future.

`WasiApp.{props,targets}`

This is for wasi-wasm projects. ILStrip becomes usable as a feature
for wasi-wasm because of this consolidation.

Current state of build files: ``` wasm: WasmApp.props, WasmApp.targets, WasmApp.Native.targets wasi: WasiApp.props, WasiApp.targets, WasiApp.Native.targets `` The wasm, and wasi build have lot of shared code but that is not representative in the actual files, since the wasi targets started life as a quick-copy-comment-out-bits of the wasm targets. This commit consolidates these into: ``` common: WasmApp.Common.props, WasmApp.Common.targets wasm : WasmApp.props, WasmApp.targets wasi : WasiApp.props, WasiApp.targets ``` ## `WasmApp.Common.{props,targets}` This has all the common parts of the build for browser-wasm, and wasi, and includes bits from `WasmApp.{props,targets}`, and `WasmApp.Native.{props,targets}`. - The top level target remains the same - `WasmBuildApp`. - There are a few "public" targets that can be hooked into: - `PrepareInputsForWasmBuild` - `WasmGenerateAppBundle` - `PrepareForWasmBuildNative` - `WasmLinkDotNet` - all these public targets have corresponding `*DependsOn` properties which can be used for extending the build note: this commit does not add a public target for AOT, but it might be added in future. ## WasmApp.{props,targets} This is for `browser-wasm` projects. The file might be renamed in future. ## WasiApp.{props,targets} This is for `wasi-wasm` projects. `ILStrip` becomes usable as a feature for `wasi-wasm` because of this consolidation.

ghost · 2023-12-08T01:18:43Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

[wasm/wasi] Consolidate build targets

Current state of build files:

    wasm: WasmApp.props, WasmApp.targets, WasmApp.Native.targets
    wasi: WasiApp.props, WasiApp.targets, WasiApp.Native.targets
``

The wasm, and wasi build have lot of shared code but that is not
representative in the actual files, since the wasi targets started life
as a quick-copy-comment-out-bits of the wasm targets.

This commit consolidates these into:

common: WasmApp.Common.props, WasmApp.Common.targets
wasm : WasmApp.props, WasmApp.targets
wasi : WasiApp.props, WasiApp.targets


 ## `WasmApp.Common.{props,targets}`

This has all the common parts of the build for browser-wasm, and wasi,
and includes bits from `WasmApp.{props,targets}`, and
`WasmApp.Native.{props,targets}`.

- The top level target remains the same - `WasmBuildApp`.
- There are a few "public" targets that can be hooked into:
    - `PrepareInputsForWasmBuild`
    - `WasmGenerateAppBundle`
    - `PrepareForWasmBuildNative`
    - `WasmLinkDotNet`

- all these public targets have corresponding `*DependsOn` properties
  which can be used for extending the build

note: this commit does not add a public target for AOT, but it might be
added in future.

 ## WasmApp.{props,targets}

This is for `browser-wasm` projects. The file might be renamed in
future.

 ## WasiApp.{props,targets}

This is for `wasi-wasm` projects. `ILStrip` becomes usable as a feature
for `wasi-wasm` because of this consolidation.


<table>
  <tr>
    <th align="left">Author:</th>
    <td>radical</td>
  </tr>
  <tr>
    <th align="left">Assignees:</th>
    <td>-</td>
  </tr>
  <tr>
    <th align="left">Labels:</th>
    <td>

`arch-wasm`

</td>
  </tr>
  <tr>
    <th align="left">Milestone:</th>
    <td>-</td>
  </tr>
</table>
</details>

radical · 2023-12-08T01:20:09Z

dotnet/emsdk#621 is needed for this.

radical · 2023-12-08T01:23:22Z

This is admittedly kinda hard to review, so I'm depending on the tests quite a bit. This does not change core specifics of the build, but moves around things, different public targets, and dependencies.

radical · 2023-12-08T01:32:53Z

TODO before merge:

test with perf pipeline (https://dev.azure.com/dnceng/internal/_build/results?buildId=2331512&view=results)

TODO for follow up PRs:

add wasi-sdk version check
runtime tests - move to using LocalBuild instead of InTree targets

src/mono/wasi/Wasi.Build.Tests/ILStripTests.cs

radical · 2023-12-08T01:47:27Z

/azp run runtime-wasm

azure-pipelines · 2023-12-08T01:47:36Z

Azure Pipelines successfully started running 1 pipeline(s).

pavelsavara · 2023-12-08T09:43:39Z

eng/testing/tests.wasm.targets

      <BundleFiles Include="$(RuntimeConfigFilePath)"           TargetDir="publish" />

      <BundleFiles Include="$(WasmSharedPath)data\aot-tests\*"  TargetDir="publish" />
+      <!-- FIXME: what would be the correct place to do this? -->


should we rename wasm -> browser (OS) and use wasm for the (architecture) ?

Yeah, I avoided those renames in this PR. We are following the pattern you suggested in most places now, like tests.{wasm,browser,wasi}.targets, and recently sendtohelix-{wasm,browser,wasi}.proj.

radical · 2023-12-08T21:07:34Z

For review, for the wasm case I think we have pretty good test coverage to be reasonably confident. There will be likely be some cases missed, or small regressions, which we can be fixed in follow up PRs.
For wasi, we have limited build support right now anyway, and some of that is covered by tests, and this PR makes it possible to have the same kinda features as the wasm build - for example, ilstrip, singlefilebundle (tbd).

src/mono/wasm/build/WasmApp.Common.targets

vargaz · 2023-12-11T12:24:58Z

Would be nice to review/merge this quickly so it doesn't conflict with other PRs, i.e.
#95834

fanyang-mono

ILStrip test looks good to me.

src/mono/wasi/Wasi.Build.Tests/ILStripTests.cs

…e-targets

lewing

Looks ready to merge then we will need to keep an eye on the builds

Blazor size regression was fixed by: ``` commit ec31705 Author: Ankit Jain <[email protected]> Date: Wed Dec 6 05:36:59 2023 -0500 [wasm] Fix regressed file sizes for blazor (dotnet#92664) ``` .. but a subsequent PR created close to that undid some of the changes: ``` commit a128c15 Author: Ankit Jain <[email protected]> Date: Mon Dec 11 15:45:58 2023 -0500 [wasm/wasi] Consolidate build targets (dotnet#95775) ``` Essentially, `-g` was being passed to the link, and compile-bc steps. Found in dotnet/perf-autofiling-issues#25891 .

* [wasm/wasi] Fix size regression Blazor size regression was fixed by: ``` commit ec31705 Author: Ankit Jain <[email protected]> Date: Wed Dec 6 05:36:59 2023 -0500 [wasm] Fix regressed file sizes for blazor (#92664) ``` .. but a subsequent PR created close to that undid some of the changes: ``` commit a128c15 Author: Ankit Jain <[email protected]> Date: Mon Dec 11 15:45:58 2023 -0500 [wasm/wasi] Consolidate build targets (#95775) ``` Essentially, `-g` was being passed to the link, and compile-bc steps. Found in dotnet/perf-autofiling-issues#25891 . * [wasm] Add flag missed in the consolidate PR

radical added 5 commits December 7, 2023 19:59

[wasi] WBT: Use TestOutputWrapper

74ee7cd

[wasm] WBT: Track target name changes

0f72775

[wasi] WBT: Add new tests

c6408c3

cleanup

25ab062

radical added the arch-wasm WebAssembly architecture label Dec 8, 2023

ghost assigned radical Dec 8, 2023

ghost added the area-Build-mono label Dec 8, 2023

radical mentioned this pull request Dec 8, 2023

Emscripten.Cache.*: Set WasmCachePath in Sdk.props instead of in a target dotnet/emsdk#621

Merged

cleanup

961df64

radical commented Dec 8, 2023

View reviewed changes

src/mono/wasi/Wasi.Build.Tests/ILStripTests.cs Show resolved Hide resolved

radical requested review from maraf and ilonatommy December 8, 2023 03:46

radical marked this pull request as ready for review December 8, 2023 03:47

radical requested review from lewing, pavelsavara, kg and marek-safar as code owners December 8, 2023 03:47

This was referenced Dec 8, 2023

[wasm][debugger] Debugger tests disabled #89409

Open

The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451

Closed

pavelsavara reviewed Dec 8, 2023

View reviewed changes

build-analysis bot mentioned this pull request Dec 8, 2023

Interop tests crash on Mono #95792

Closed

pavelsavara mentioned this pull request Dec 10, 2023

[NativeAOT LLVM] Integrate dotnet.js into ILCompiler.LLVM dotnet/runtimelab#2453

Merged

pavelsavara mentioned this pull request Dec 11, 2023

[mono][wasm] Move some runtime code to a separate runtime.h/runtime.c… #95834

Merged

ilonatommy approved these changes Dec 11, 2023

View reviewed changes

src/mono/wasm/build/WasmApp.Common.targets Outdated Show resolved Hide resolved

src/mono/wasm/build/WasmApp.Common.targets Show resolved Hide resolved

fanyang-mono approved these changes Dec 11, 2023

View reviewed changes

src/mono/wasi/Wasi.Build.Tests/ILStripTests.cs Outdated Show resolved Hide resolved

radical added 3 commits December 11, 2023 13:18

Merge remote-tracking branch 'origin/main' into wasm-build-consolidat…

cc221c8

…e-targets

address feedback

b6914c9

address feedback

260bc0b

lewing approved these changes Dec 11, 2023

View reviewed changes

lewing merged commit a128c15 into dotnet:main Dec 11, 2023

radical deleted the wasm-build-consolidate-targets branch December 11, 2023 20:46

radical mentioned this pull request Dec 11, 2023

[wasm] Add WasmSingleFileBundle for browser-wasm projects too #95480

Closed

github-actions bot locked and limited conversation to collaborators Jan 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[wasm/wasi] Consolidate build targets #95775

[wasm/wasi] Consolidate build targets #95775

Uh oh!

radical commented Dec 8, 2023 •

edited

Loading

Uh oh!

ghost commented Dec 8, 2023

Uh oh!

radical commented Dec 8, 2023

Uh oh!

radical commented Dec 8, 2023

Uh oh!

radical commented Dec 8, 2023 •

edited

Loading

Uh oh!

Uh oh!

radical commented Dec 8, 2023

Uh oh!

azure-pipelines bot commented Dec 8, 2023

Uh oh!

pavelsavara Dec 8, 2023

Uh oh!

radical Dec 8, 2023

Uh oh!

radical commented Dec 8, 2023

Uh oh!

Uh oh!

Uh oh!

vargaz commented Dec 11, 2023

Uh oh!

fanyang-mono left a comment

Uh oh!

Uh oh!

lewing left a comment

Uh oh!

Uh oh!

[wasm/wasi] Consolidate build targets #95775

[wasm/wasi] Consolidate build targets #95775

Uh oh!

Conversation

radical commented Dec 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

WasmApp.Common.{props,targets}

WasmApp.{props,targets}

WasiApp.{props,targets}

Uh oh!

ghost commented Dec 8, 2023

Uh oh!

radical commented Dec 8, 2023

Uh oh!

radical commented Dec 8, 2023

Uh oh!

radical commented Dec 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO before merge:

TODO for follow up PRs:

Uh oh!

Uh oh!

radical commented Dec 8, 2023

Uh oh!

azure-pipelines bot commented Dec 8, 2023

Uh oh!

pavelsavara Dec 8, 2023

Choose a reason for hiding this comment

Uh oh!

radical Dec 8, 2023

Choose a reason for hiding this comment

Uh oh!

radical commented Dec 8, 2023

Uh oh!

Uh oh!

Uh oh!

vargaz commented Dec 11, 2023

Uh oh!

fanyang-mono left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lewing left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

radical commented Dec 8, 2023 •

edited

Loading

`WasmApp.Common.{props,targets}`

`WasmApp.{props,targets}`

`WasiApp.{props,targets}`

radical commented Dec 8, 2023 •

edited

Loading