Update GLTF source generator to .NET 5 final specs

[paulbartrum]

I get build errors when building the GltfLoader project with the final .NET 5.0 SDK installed:

Warning: Method 'Initialize' in type 'GltfGenerator.Tasks.GltfSourceGenerator' from assembly 'GltfGenerator, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.
Error CS0234: The type or namespace name 'Schema' does not exist in the namespace 'GltfLoader'
   .... <lots more similar errors> ...

I'm not sure what is causing the source generator to fail exactly, but I saw that the final .NET 5 version of source generators made several breaking changes to the source generators spec. So I took the opportunity to update to the final version of the spec for the GltfGenerator and GltfLoader projects.

I tried updating the Shaders.Tasks project too, but it seems that local project references are problematic in a source generator, so the Tasks project would need to be combined with the Shaders project. That's quite a bit of disruption, so I'll just contribute the Gltf changes for now.

[Sohra]

Hey Paul, I like what you've done here and closed my PR as superceded by this.

This however didn't work for me out-of-the-box and I propose some changes and tidyup for your consideration.

[Sohra]

Sorry if my earlier edits on that message were generating emails to anyone. I made a mess of it.

I want to separate my comments regarding applying this same pattern to the project DirectX12GameEngine.Shaders.Tasks, which I subsequently had a go at doing as well.

I followed your (Paul's) pattern on this PR, and applying Eric StJohn's solution instead of the GeneratePathProperty solution (referred to below in my comments), DirectX12GameEngine.Shaders.Tasks appeared to be work, although upon closer inspection, no, I hit a coupld of build warnings on DirectX12GameEngine.Rendering:

CS8034	Unable to load Analyzer assembly \DirectX12GameEngine\DirectX12GameEngine.Shaders\bin\Debug\netstandard2.0\DirectX12GameEngine.Shaders.dll: Could not find file '\DirectX12GameEngine\DirectX12GameEngine.Shaders\bin\Debug\netstandard2.0\DirectX12GameEngine.Shaders.dll'.
CS8034	Unable to load Analyzer assembly \DirectX12GameEngine\DirectX12GameEngine.Shaders.Tasks\bin\Debug\netstandard2.0\DirectX12GameEngine.Shaders.Tasks.dll: Could not find file '\DirectX12GameEngine\DirectX12GameEngine.Shaders.Tasks\bin\Debug\netstandard2.0\DirectX12GameEngine.Shaders.Tasks.dll'.

Is this the same issue you encountered, or was it something else?

It seems despite the pattern working to pull in the transitive dependencies of package references found in GltfGenerator, it's not working the same way for transitive dependencies of project references - despite TargetPathWithTargetPlatformMoniker including the dll reference just like it does with all the others from NuGet packages.

[paulbartrum]

Your suggestions worked fine for me @Sohra, and you've obviously put a bunch of effort into researching this, so I've applied pretty all your suggestions.

Is this the same issue you encountered, or was it something else?

That was it, yup. I'm used to dependencies "just working" in .NET so finding out there's all these dependency issues with analyzers is quite disappointing. Here's hoping this gets better in future releases.

[Sohra]

Update on the Shader.Tasks project... I came across dotnet/roslyn#48746 (reply in thread) tonight and following that, switching to a release build, the project built successfully...
Switching back to debug again, continues to build, but one would guess it must be intermittently, based on @onetsmr experience...

If you want to have a go, my final Shader.Tasks project content was:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
    <LangVersion>8.0</LangVersion>
    <Nullable>enable</Nullable>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
  </PropertyGroup>

  <ItemGroup Label="Package references needed for source generators">
    <PackageReference Include="Microsoft.CodeAnalysis.CSharp.Workspaces" Version="3.9.0" PrivateAssets="all" />
    <PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="3.3.2" PrivateAssets="all" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\DirectX12GameEngine.Shaders\DirectX12GameEngine.Shaders.csproj" PrivateAssets="all" />
  </ItemGroup>

  <PropertyGroup>
    <GetTargetPathDependsOn>$(GetTargetPathDependsOn);GetDependencyTargetPaths</GetTargetPathDependsOn>
  </PropertyGroup>

  <Target Name="GetDependencyTargetPaths" DependsOnTargets="ResolveReferences">
    <ItemGroup>
      <TargetPathWithTargetPlatformMoniker Include="@(ReferenceCopyLocalPaths)" Condition="('%(ReferenceCopyLocalPaths.Extension)' == '.dll') And !$([System.String]::Copy(%(ReferenceCopyLocalPaths.FullPath)).Contains('\native\'))" IncludeRuntimeDependency="false" />
    </ItemGroup>
  </Target>

</Project>

Note TargetPathWithTargetPlatformMoniker has a Condition on it unlike in the GltfGenerator project - this is to filter out the PDBs and native dlls that ReferenceCopyLocalPaths contains but TargetPathWithTargetPlatformMoniker can't handle. Without doing this, you get an error something along the lines of PE image doesn't contained managed metadata

Remember to update the three consuming applications that project reference this (add OutputItemType="Analyzer" to the project reference, delete the Analyzer item group, and change LangVersion to 8.0)

Update: If you encounter an error in DirectX12ComputeShaderSample missing a reference to Vortice.Direct3D12, this could be resolved by removing PrivateAssets="compile" from the Graphic's project's package reference to it, so that it will be inherited by the consuming projects (declaring the compile assets private stops this occurring).

[JuandreG]

I'm looking at this project and I am unable to build it. I tried these solutions but had no success. I am very interested in this, anything I can try to get this to build? Is there a way we can skip the source generators and just have the files needed for the build, or does it still generate files at runtime?

@Sohra You mentioned that you made nuget packages of these projects, this is me showing interest in those packages.

[Sohra]

I'm looking at this project and I am unable to build it. I tried these solutions but had no success. I am very interested in this, anything I can try to get this to build? Is there a way we can skip the source generators and just have the files needed for the build, or does it still generate files at runtime?

@Sohra You mentioned that you made nuget packages of these projects, this is me showing interest in those packages.

Hi @JuandreG,
If you take the changes in this PR, it should solve the build issues for the Gltf projects, but what @paulbartrum didn't fix was the same ones for the DirectX12GameEngine.Shader.Tasks project. To fix those, follow my advice at #12 (comment) .

Although as I check this myself today, I actually have made further changes since that message was written (message above definition on left, and my local changes on the right):

I closed my PR in favour of Paul's as he did a nicer job, he just didn't apply the same fixes to the second source generator. Apply the project changes in my earlier comment, and the further change in the above screenshot if necessary, along with updating all the projects that consume this second source generator (adding the OutputItemType="Analyser" on the project reference). Finally these two lines need to change in the ShaderSourceGenerator for compatibility with the current source generator APIs:

That should get the whole repo into a building state for you, at least assuming your on Visual Studio 17.6.5 like I currently am.

This library was a bit "heavy" for my use case so I didn't want to fork the repo and publish my own NuGet packages for it. It's a shame the original author hasn't found time to make the above changes, and merge this PR and publish some NuGet packages for others to use and benefit from.

What I like the most about this is the .NET to HLSL shader compilation, and ShaderGeneratorContext class that helps with the generation of corresponding root signatures and pipeline states. I've been playing around with these over at https://github.com/Sohra/DirectX-Graphics-Samples/ which applies concepts from this repo, upon some of the Microsoft DirectX samples ported from C++ to C#.

[JuandreG]

I added the changes you made before I commented here, I also updated the DirectX12GameEngine.Shader.Tasks project. It is still not building. 46 Errors and 42 Warnings.

Here are some of the errors from GltfLoader:

The type or namespace name 'BufferView' could not be found
The type or namespace name 'Gltf' could not be found
The type or namespace name 'Image' could not be found
The type or namespace name 'Schema' could not be found
The type or namespace name 'Schema' does not exist in 'GltfLoader'

It is also complaining about missing dlls:

GltfLoader.dll
DirectX12GameEngine.Physics.dll
DirectX12GameEngine.Engine.dll
DirectX12GameEngine.Assets.dll

But they are in the solution, I did not remove them or anything. Only changes I made was the ones you and paulbartrum suggested.

a Warning I'm seeing is:

Generator 'GltfSourceGenerator' failed to generate source. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'FileNotFoundException' with message 'Could not load file or assembly 'System.Text.Json, Version=5.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.'

I am on Visual Studio 17.5.3 but I don't think this is the issue.

[JuandreG]

I was hoping to use the "3D Viewer" part of this as it looks really good. I am planning on building a type of designer where the rendering should look good. But it seems Helixtoolkit could be a viable alternative. It has some drawbacks but I will find a way around it.

Thanks for your inputs and for trying to help. I think I will now give up on trying to get this to work, it is taking up too much time.

[Sohra]

Did you remove the lines:

  <ItemGroup Label="External dependencies">
    <PackageReference Include="System.CodeDom" Version="5.0.0" PrivateAssets="all" />
    <PackageReference Include="System.Text.Json" Version="5.0.1" PrivateAssets="all" />
  </ItemGroup>

from the Shader.Tasks project file per my above screenshot? Oh wait a minute, I think I gave you misinformation earlier... I think you need to add these. I have like 3 copies of this repo checked out on my end and I'm getting a bit mixed up, sorry. What if you try adding these, and removing the PrivateAssets="all" from the Shaders ProjectReference?

If that still doesn't work for you - first clean out your Intellisense DB (i.e. Close the solution, delete the .vs folder at the solution root) and delete and bin and obj folders in each project... then reopen the solution and attempt to build.

If this still doesn't work... I've pushed a branch for you that applies all of these changes and is building fine for me, although I haven't tried running it again. It was a couple of years ago I was doing this stuff now (;

I hope this helps @JuandreG

[Sohra]

I just tested the DirectX12WinFormsApp project runs on the branch I gave you before... by "3D Viewer" did you mean the DirectX12GameEngine.Editor project? That's not building for me and I don't have to investigate why sorry, but I'd guess because its a UWP app... and that whole platform has kinda been abandoned by Microsoft so if you need this project, then I advise trying to port it over to a WinForms app instead, running on .NET 6 (all those .NET Standard 2.0 class libraries should run fine on .NET 6).

[JuandreG]

I also just tried running the one you pushed with no success, we are probably experiencing the same issue. By "3D Viewer" I mean everything needed from this project to just view a 3D .glb file.

I would have liked to use UWP, I know it gets a lot of hate but I have had success with it.

Anyway, thanks so much for your trouble. I really appreciate it!

[Sohra]

If you are willing to use the project I named before, I should be able to help you get it building and running like so, at least:

This means you can salvage the GLB model loading and rendering from this repo for your designer, as that's working just fine.

My C# ports of the Microsoft C++ DirectX samples are using parts of this library on .NET 6 with newer versions of the Vortice library if you are interested, but I didn't bring the GLB projects. I'm using an old DirectX 9-era ".X" model format over there.

However I honestly think you are wasting your time with UWP projects. I too am very sad about the fate of UWP as I was all in for WinRT right from the Windows 8 days. Back in 2021, I was also trying to port these projects to .NET 5 and WinUI 3 but I finally gave up (there's another branch I pushed last night of my efforts on that over on my fork of this repo, but it doesn't look like I got far).

I would suggest you keep in mind the support timeframes for all these technologies, and avoid building something new on obsolete technology. .NET Core 3.0 which this repo was using is out of support, as is 3.1, and even .NET 5 is no longer supported. We should really be working in .NET 6 or higher today. https://dotnet.microsoft.com/en-us/platform/support/policy is a handy page to check the support timeframes (and it has a link that provides this same data for the out of support releases too). WinRT also passed its end of support date back in January this year - see https://learn.microsoft.com/en-us/lifecycle/products/windows-rt

Now I haven't followed whether there's been any developments since 2021, but at the time the announcement was made that .NET 5 and .NET 6 would not be brought to UWP, and I suspect that's still the case. Microsoft published guidance that they still update on porting UWP to alternate technologies, see https://learn.microsoft.com/en-us/windows/apps/windows-app-sdk/migrate-to-windows-app-sdk/migrate-to-windows-app-sdk-ovw and microsoft/WindowsAppSDK#1615

If you want some help to get the app running per the above screenshot that I just took from the branch I pushed for you, on your side, then let me know - maybe we can setup a screenshare or something to figure out what's going wrong. Otherwise, good luck with building your designer, whichever path you take (:

[JuandreG]

Hi @Sohra.

Thank you so much for your feedback, sorry for the late reply.

I also got the WinForms project to build and run, the mouse events on it is a bit weird but that is besides the point, I did get it to run.

Though I like what I saw I already started exploring the HelixToolkit. Way less of a headache to get everything working on WinUI and/or UWP plus it has much of what I need already built into the api so it is actually a no-brainer. If you have some experience with it we can collaborate? I need all the help I can get, it seems it is quite a large api.

For now I am still using UWP but you make great points, when I'm done exploring the HelixToolkit and I have all the answers to my questions I will move over to a WinUI project. The 2 work very similarly so moving over should not be a problem.

Thanks for all your inputs, it is greatly appreciated,

[paulbartrum]

Seems like this project is dead :-(