[Helix] Shared framework support + Templates tests #19177

HaoK · 2020-02-20T03:53:02Z

Rebased version of #16828

jkotalik · 2020-02-20T19:29:36Z

@HaoK do you still need me to look at why IIS tests are failing here now?

HaoK · 2020-02-20T19:32:40Z

Yeah, I'm close to having an initial set of the templates tests working ( the easy ones), but we'll need to figure out why the IIS tests are failing before this PR can be merged. The templates tests are failing on the regular runs too right now, so something in the build might be messed up

eng/helix/content/InstallAppRuntime.ps1

.azure/pipelines/ci.yml

HaoK · 2020-02-22T19:03:59Z

Getting closer, looks like a few template tests are failing on mac and win7 and not everywhere else, mac failure looks to be cert related? @Tratcher can you help route these?

Full mac helix log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-aspnetcore-refs-pull-19177-merge-a10cb06b57c145b58a/ProjectTemplates.Tests--netcoreapp5.0/console.43e17a96.log?sv=2019-02-02&se=2020-03-03T11%3A48%3A00Z&sr=c&sp=rl&sig=OscPpILIMxefxFtC8fpRLw1BaBPp3d7TKq%2BP9VNHCYQ%3D

 Waiting until ASP.NET application is accepting connections...
 \x1b[40m\x1b[37mdbug\x1b[39m\x1b[22m\x1b[49m: Microsoft.Extensions.Hosting.Internal.Host[1]
       Hosting starting
 \x1b[40m\x1b[37mdbug\x1b[39m\x1b[22m\x1b[49m: Grpc.AspNetCore.Server.Model.Internal.ServiceRouteBuilder[2]
       Discovering gRPC methods for AspNet.grpc.oijlux20ggo.GreeterService.
 \x1b[40m\x1b[37mdbug\x1b[39m\x1b[22m\x1b[49m: Grpc.AspNetCore.Server.Model.Internal.ServiceRouteBuilder[1]
       Added gRPC method 'SayHello' to service 'greet.Greeter'. Method type: 'Unary', route pattern: '/greet.Greeter/SayHello'.
 \x1b[40m\x1b[37mdbug\x1b[39m\x1b[22m\x1b[49m: Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer[2]
       Failed to locate the development https certificate at '(null)'.
 \x1b[40m\x1b[37mdbug\x1b[39m\x1b[22m\x1b[49m: Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer[1]
       Unable to locate an appropriate development https certificate.
 \x1b[41m\x1b[1m\x1b[37mcrit\x1b[39m\x1b[22m\x1b[49m: Microsoft.AspNetCore.Server.Kestrel[0]
       Unable to start Kestrel.
 System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found or is out of date.
 To generate a developer certificate run 'dotnet dev-certs https'. To trust the certificate (Windows and macOS only) run 'dotnet dev-certs https --trust'.
 For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.
    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions, Action`1 configureOptions)
    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions)
    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(AddressBindContext context)
    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IServerAddressesFeature addresses, KestrelServerOptions serverOptions, ILogger logger, Func`2 createBinding)
    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
 [ERROR] Unhandled exception. System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found or is out of date.
 [ERROR] To generate a developer certificate run 'dotnet dev-certs https'. To trust the certificate (Windows and macOS only) run 'dotnet dev-certs https --trust'.
 [ERROR] For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.
 [ERROR]    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions, Action`1 configureOptions)
 [ERROR]    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions)
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(AddressBindContext context)
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IServerAddressesFeature addresses, KestrelServerOptions serverOptions, ILogger logger, Func`2 createBinding)
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
 [ERROR]    at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
 [ERROR]    at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
 [ERROR]    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
 [ERROR]    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
 [ERROR]    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
 [ERROR]    at AspNet.grpc.oijlux20ggo.Program.Main(String[] args) in /private/tmp/helix/working/AD850971/w/A902090F/e/Templates/BaseFolder/AspNet.grpc.oijlux20ggo/Program.cs:line 15

Grpc failure on Win7 looks like its just not supported? Does that template test need to be conditioned on something?

 System.NotSupportedException: HTTP/2 over TLS is not supported on Windows 7 due to missing ALPN support.
    at Microsoft.AspNetCore.Server.Kestrel.Https.Internal.HttpsConnectionMiddleware..ctor(ConnectionDelegate next, HttpsConnectionAdapterOptions options, ILoggerFactory loggerFactory)
    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.<>c__DisplayClass12_0.<UseHttps>b__0(ConnectionDelegate next)
    at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.Build()
    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.<>c__DisplayClass21_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
 --- End of stack trace from previous location ---
    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context)
    at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context)
    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(AddressBindContext context)
    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IServerAddressesFeature addresses, KestrelServerOptions serverOptions, ILogger logger, Func`2 createBinding)
    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
 [ERROR] Unhandled exception. System.NotSupportedException: HTTP/2 over TLS is not supported on Windows 7 due to missing ALPN support.
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Https.Internal.HttpsConnectionMiddleware..ctor(ConnectionDelegate next, HttpsConnectionAdapterOptions options, ILoggerFactory loggerFactory)
 [ERROR]    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.<>c__DisplayClass12_0.<UseHttps>b__0(ConnectionDelegate next)
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.Build()
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.<>c__DisplayClass21_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
 [ERROR] --- End of stack trace from previous location ---
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context)
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context)
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(AddressBindContext context)
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IServerAddressesFeature addresses, KestrelServerOptions serverOptions, ILogger logger, Func`2 createBinding)
 [ERROR]    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
 [ERROR]    at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
 [ERROR]    at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
 [ERROR]    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
 [ERROR]    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
 [ERROR]    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
 [ERROR]    at AspNet.grpc._4wm2a5ilva3.Program.Main(String[] args) in C:\h\w\A4FE0892\w\ACEB0953\e\Templates\BaseFolder\AspNet.grpc.4wm2a5ilva3\Program.cs:line 15

Tratcher · 2020-02-22T20:44:23Z

@HaoK gRPC isn't supported on MAC or Win7. They already have built in skips:

aspnetcore/src/ProjectTemplates/test/GrpcTemplateTest.cs

Lines 42 to 55 in b296f0c

    
           // These templates are HTTPS + HTTP/2 only which is not supported on Mac due to missing ALPN support. 
        
           // https://github.com/dotnet/aspnetcore/issues/11061 
        
           if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX)) 
        
           { 
        
               Assert.True(serverProcess.Process.HasExited, "built"); 
        
               Assert.Contains("System.NotSupportedException: HTTP/2 over TLS is not supported on macOS due to missing ALPN support.", 
        
                   ErrorMessages.GetFailedProcessMessageOrEmpty("Run built service", Project, serverProcess.Process)); 
        
           } 
        
           else if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows) && Environment.OSVersion.Version < new Version(6, 2)) 
        
           { 
        
               Assert.True(serverProcess.Process.HasExited, "built"); 
        
               Assert.Contains("System.NotSupportedException: HTTP/2 over TLS is not supported on Windows 7 due to missing ALPN support.", 
        
                   ErrorMessages.GetFailedProcessMessageOrEmpty("Run built service", Project, serverProcess.Process)); 
        
           }

The Win7 error matches the expected failure in the test, not sure why it didn't handle it.

The MacOS failure is interesting, it's missing the dev cert. It's not clear from the test code how it's supposed to provision the dev cert.

@JunTaoLuo @JamesNK @javiercn

HaoK · 2020-02-22T22:05:06Z

Guess those RuntimeInfo checks aren't working well on helix, I'll just slap on SkipOnHelix queues for the specific queues for the grpc tests

JamesNK · 2020-02-22T22:30:19Z

I don't know if it is related but we've recently changed the gRPC template's launchSettings.json to include an HTTP endpoint to help macOS devs.

#19252

JamesNK · 2020-02-22T22:32:25Z

Hmm, I wonder why running the template test on macOS passed after that update.

I thought it only gave the error "HTTP/2 over TLS is not supported on macOS due to missing ALPN support." if HTTPS was the only endpoint.

HaoK · 2020-02-24T16:08:06Z

Ok looks like we have a stable initial set of template tests working, there are still a bunch disabled, so I kept the azdo job running the template tests as well until they are full switched over. This is an initial checkpoint that I'd like to get merged though, since it brings over the shared framework, and runs most of the simple template tests. The ones that use identity and razor compilation are still failing due to restore issues (EF/compilation dll). I'll file issues to track re enabling the rest of those tests. But might as well start getting this reviewed

.azure/pipelines/ci.yml

dougbu · 2020-02-25T03:45:31Z

src/ProjectTemplates/test/Helpers/Project.cs

+            ? GetAssemblyMetadata("ArtifactsLogDir")
+            : Path.Combine(Environment.GetEnvironmentVariable("HELIX_DIR"), "logs");
+
+        // FIGURE OUT EF PATH


File an issue tracking this

#19327 is the meta issue tracking all of the various skipped/remaining work

dougbu · 2020-02-25T04:17:34Z

.azure/pipelines/ci.yml

@@ -655,15 +657,19 @@ stages:

  - template: jobs/default-build.yml
    parameters:
-      condition: ne(variables['Build.Reason'], 'PullRequest')
+      # condition: ne(variables['Build.Reason'], 'PullRequest') UNCOMMENT after testing


Is this bit ready to review?

Yes, the only way to test the daily/arm queues is by commenting out the conditions, I'll undo them once everything is ready to go

HaoK · 2020-02-26T16:23:55Z

@dougbu are you good with merging this once everything is green, I will undo the daily and arm conditions as well before merging

dougbu

Yes, this looks fine modulo the two things you said you'd undo before merging

HaoK · 2020-02-26T20:56:07Z

Cool merged this initial checkpoint, I'll open a new PR

HaoK mentioned this pull request Feb 20, 2020

Add support for running helix tests against built shared fx #16828

Closed

Pilchie added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Feb 20, 2020

HaoK commented Feb 20, 2020

View reviewed changes

eng/helix/content/InstallAppRuntime.ps1 Outdated Show resolved Hide resolved

jkotalik reviewed Feb 20, 2020

View reviewed changes

.azure/pipelines/ci.yml Outdated Show resolved Hide resolved

HaoK force-pushed the haok/helix-sfx2 branch 2 times, most recently from 47cef0c to c3aeed1 Compare February 21, 2020 21:04

HaoK force-pushed the haok/helix-sfx2 branch from fda3c00 to 222cf33 Compare February 24, 2020 05:35

HaoK marked this pull request as ready for review February 24, 2020 16:08

HaoK requested a review from a team February 24, 2020 16:08

dougbu reviewed Feb 25, 2020

View reviewed changes

HaoK added 12 commits February 25, 2020 23:10

Rebased PR

9146931

Update Helix.targets

3f741ec

Update ReactReduxTemplateTest.cs

889e5f4

Skip symbols nupkgs

0648c52

Update BlazorServerTemplateTest.cs

1493749

Don't run template tests on helix build step

befd44e

Update ProjectTemplates.Tests.csproj

9d7fe51

remove nobuild

6f3e088

Update ci.yml

b566bb7

Update ci.yml

5fc3498

Update RazorPagesTemplateTest.cs

79b61ce

Update ProcessEx.cs

a18843a

HaoK added 22 commits February 25, 2020 23:10

Update RazorPagesTemplateTest.cs

da2b885

Update EmptyWebTemplateTest.cs

2a5462b

Update WebApiTemplateTest.cs

80aad57

Update ci.yml

3bb408b

Try -buildNative

b0f2499

Update InstallAppRuntime.ps1

9392fc6

Update MvcTemplateTest.cs

efeb977

Update ci.yml

1467920

Update ci.yml

770288a

Update ci.yml

ef4009d

Update ci.yml

5628056

Try reenabling GRPC after rebase

730784f

Skip template tests on arm64 for now

d1cbb02

Explicitly run template tests for helix jobs

5c4d14f

Add templates to quarantine run

594eddf

Update ci.yml

a729730

Grpc still broken

3ce3058

Update ProjectTemplates.Tests.csproj

84cac20

Take rid as parameter

a981292

Pass rid as parameter for sfx

1472dd7

Take rid

8a99b9f

Remove commented out line

70ad733

HaoK force-pushed the haok/helix-sfx2 branch from 60c0a06 to 70ad733 Compare February 26, 2020 07:10

dougbu approved these changes Feb 26, 2020

View reviewed changes

Restore PR checks

7fc471f

HaoK merged commit 4498058 into master Feb 26, 2020

HaoK deleted the haok/helix-sfx2 branch February 26, 2020 20:55

HaoK mentioned this pull request Feb 26, 2020

Helix: make the aspnetcore shared framework available in Helix tests #11128

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Helix] Shared framework support + Templates tests #19177

[Helix] Shared framework support + Templates tests #19177

HaoK commented Feb 20, 2020

jkotalik commented Feb 20, 2020

HaoK commented Feb 20, 2020

HaoK commented Feb 22, 2020

Tratcher commented Feb 22, 2020

HaoK commented Feb 22, 2020

JamesNK commented Feb 22, 2020

JamesNK commented Feb 22, 2020

HaoK commented Feb 24, 2020

dougbu Feb 25, 2020

HaoK Feb 25, 2020

dougbu Feb 25, 2020

HaoK Feb 25, 2020

HaoK commented Feb 26, 2020

dougbu left a comment

HaoK commented Feb 26, 2020

[Helix] Shared framework support + Templates tests #19177

[Helix] Shared framework support + Templates tests #19177

Conversation

HaoK commented Feb 20, 2020

jkotalik commented Feb 20, 2020

HaoK commented Feb 20, 2020

HaoK commented Feb 22, 2020

Tratcher commented Feb 22, 2020

HaoK commented Feb 22, 2020

JamesNK commented Feb 22, 2020

JamesNK commented Feb 22, 2020

HaoK commented Feb 24, 2020

dougbu Feb 25, 2020

Choose a reason for hiding this comment

HaoK Feb 25, 2020

Choose a reason for hiding this comment

dougbu Feb 25, 2020

Choose a reason for hiding this comment

HaoK Feb 25, 2020

Choose a reason for hiding this comment

HaoK commented Feb 26, 2020

dougbu left a comment

Choose a reason for hiding this comment

HaoK commented Feb 26, 2020