Is runtime.linux-x64.Microsoft.NETCore.Runtime.ObjWriter necessary for source-build?

[crummel]

This is a new prebuilt in 7.0 that we're seeing being used by coreclr/ILCompiler. Is this a component that needs to be included in source-build? It seems to be related to Native AOT but there's also some notes where it's being used that make it seem testing-related.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[jkotas]

objwriter is not testing related. It needs to be included in source-build, assuming we want NativeAOT packages to be produced by source-build.

[crummel]

I think that means we need to include https://github.com/dotnet/llvm-project in source-build as well, correct?

cc @MichaelSimons

[hoyosjs]

Yes

[agocke]

I think this is an important question:

assuming we want NativeAOT packages to be produced by source-build.

There have been some other design problems with NativeAOT and the source build restrictions. In particular, NativeAOT uses pre-built static libraries from the runtime to compile packages. If we use the source build version, things break because they reference specific versions of openssl instead of dynamically loading what's available. See #66859. There seem to be some fundamental conflicts between source build and NativeAOT in terms of compilation model.

[jkotas]

There have been some other design problems with NativeAOT and the source build restrictions.

Source build today conflates building from a big pile of sources with applying distro specific policies (#66859 is caused by distro specific policy for dynamic linking). It means that the source build produces product with different characteristics than regular build. It is by design.

I do not think that these are fundamental conflicts. #66859 can be fixed by creating .props file with the additional libraries to reference when the distro specific policies are in affect. It is still work that would have to do be done.

[jkotas]

@tmds @omajid Any thoughts about source build and NativeAOT?

[omajid]

I agree with the comment that source-build - as it currently exists - mixes both "build everything from source" as well as "use the common Linux distribution practices" (including dynamic linking). There's no real reason that both have to be tied together. I could see someone using source-build to build on an "old" platform like CentOS 7 (and leave out the distribution policies) to get something close to Microsoft's portable builds of .NET SDK.

The OpenJDK packages in Fedora are actually exploring something like this (build from source, but violate distro policies and bundle everything), and there's a long discussion thread about that here: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/IR5C3YVVABP6LVCXQL3UY5VZPSYWDAF5/. So it's not totally out of the world that source-build could be used this way too.

I would much prefer that the source-build SDK have the same set of (possible) features as the Microsoft-built SDK, including NativeAOT.

Taking a step back, how does NativeAOT deal with bundled vs system libraries (src/native/external/) like brotli, libunwind and zlib? Could it's OpenSSL component use the same strategy with some tweaks?

I think that means we need to include https://github.com/dotnet/llvm-project in source-build as well, correct?

Is there no way to use an upstream llvm directly? Rebuilding all of llvm, aside from taking a very long time every build, is also duplicating the efforts of each distro's existing llvm packages.

[jkotas]

Is there no way to use an upstream llvm directly? Rebuilding all of llvm, aside from taking a very long time every build, is also duplicating the efforts of each distro's existing llvm packages.

We are using llvm with custom changes: https://github.com/dotnet/llvm-project/tree/objwriter/12.x. We are building a subset, not the whole thing.