Skip to content

Update MessagePack to v2.0 for SignalR #18133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Jan 24, 2020
Merged

Update MessagePack to v2.0 for SignalR #18133

merged 26 commits into from
Jan 24, 2020

Conversation

tebeco
Copy link
Contributor

@tebeco tebeco commented Jan 5, 2020

Summary of the changes (Less than 80 chars)
As the API from MessagePack v1.x that tended to be "Allocation prone" were reworked in v2.0.

Related to #18074

As said in the issue i might have to rebase the changes on master :(

I add lots of // REVIEW in the code as reviewer here probably have better answer than me :D

Out of curiosity, I change the remote URI for the submodule, as it was pointing to aspnet/MessagePack-CSharp (security reason ?) than was a fork from @AArnott repo, that is also a fork from the original repository

@tebeco tebeco changed the title Attempting to update msgpack Attempting to update MessagePack to v2.0 Jan 5, 2020
@tebeco
Copy link
Contributor Author

tebeco commented Jan 5, 2020

@pranavkm @AArnott
As i see in the history you previously worked with MessagePack, i'm not sure how the handle the current error i see from the build :

Auto-implemented property 'MessagePackReader.CancellationToken' must be fully assigned before control is returned to the caller

This seem to be caused by :
public CancellationToken CancellationToken { get; set; }

present here :
https://github.com/neuecc/MessagePack-CSharp/blob/fb55814f0c27cff2405401ebff3af1ce69344adc/src/MessagePack.UnityClient/Assets/Scripts/MessagePack/MessagePackWriter.cs#L38-L58

  • No value defaulted for the property
  • The two constructor never assign this property
  • The type is not partial, so i did not forget yet another file (or did i ?)
  • no call to the parameterless ctor to use the default value for un-initialized property

As it's a submodule, i have no idea how to leverage on that ?
Should it be fixed in @neuecc original repo ? or in the aspnet/MessagePack-CSharp ?

@AArnott,
do you know how both fork should/could be updated to pull down the branch v2.0 :
from https://github.com/neuecc/MessagePack-CSharp
to https://github.com/AArnott/MessagePack-CSharp

then to https://github.com/aspnet/MessagePack-CSharp
(not sure why the double fork here ^^)

@BrennanConroy
Copy link
Member

Couple things:
Don't mess with the fork/submodule.
And rebase the PR on master.

@tebeco
Copy link
Contributor Author

tebeco commented Jan 5, 2020

hello @BrennanConroy

Don't mess with the fork/submodule.

You mean ? i should not edit the files inside the SubModule folder i guess ? yeah, i did not
Thought the SubModule depends on 2 Fork chained to each others.
I change the SubModule URL to points directly to the source just to be able to build against the source code.

My guess is that the fork aspnet/MessagePack-CSharp exists for security reason, in order to avoid the code being tempered in the original repo ?
So yes i'll point back to the aspnet/MessagePack-CSharp once a commit for 2.0.x will be there

And rebase the PR on master.

To be honest i did it here first to know if it could be part of 3.1, i'm not sure if it's concidered as a Breaking change or not to be honest.
It is one if we concider that existing application use both

  • SignalR + MessagePack
  • MessagePack it self directly in their code

As you said i'll try tomorrow to rebase on master ASAP yes

regarding the build error :

Auto-implemented property 'MessagePackReader.CancellationToken' must be fully assigned before control is returned to the caller

i just made the change locally to both MessagePackReader (two constructor) and MessagePackWriter (two constructor) in order to run the full build locally, this seems to properpy fix the build.cmd at the root level of the repository

@tebeco
Copy link
Contributor Author

tebeco commented Jan 5, 2020

also, in order to fix the potential breaking change due to consumer using MessagePack vA on one hand and SignalR using MessagePack vB on the other hand
Could this be concidered to be part of SignalR before the release of .Net 5 ? :
#18074 (comment)

If you are careful with your assembly load contexts, you can get .NET Core to load your MessagePack v2 code in an isolated assembly load context so that it doesn't disrupt SignalR's loading of MessagePack v1 in another assembly load context. This may work well if your messagepack code is fairly isolated

I have no idea of what it implies / the change / the performance impact etc ...
So this might totally be a big NO.

To be fair I started the PR seeing that v2.x as a potential performance gain here. I wanted to see if i could run the benchs from this repo itself to re-use something existing already that could benefits to other people.

After load test with ~5000 call per second for few minutes we saw that v1 has few allocation that could be enhanced with this bump
Do you know if it's possible to locally build this branch to produce something we could use as PackageReference/FrameworkReference on a private feed so we could try to load test our bench with this build to be able to make a comparison ?
(if it's a custom FrameworkReference, it seems out of my league, never worth a try ^^)

@BrennanConroy
Copy link
Member

My guess is that the fork aspnet/MessagePack-CSharp exists for security reason, in order to avoid the code being tempered in the original repo ?

It exists because Blazor uses MessagePack internally, but since Blazor is in the shared framework and we don't put third-party libraries in the shared framework, it needs to build MessagePack from source.

So yes i'll point back to the aspnet/MessagePack-CSharp once a commit for 2.0.x will be there

I don't see why you need to do anything with aspnet/MessagePack-CSharp, it's unrelated to SignalR's usage of MessagePack.

if it could be part of 3.1, i'm not sure if it's concidered as a Breaking change

It is a breaking change to update non-patch versions of dependencies.

@tebeco
Copy link
Contributor Author

tebeco commented Jan 5, 2020

I don't see why you need to do anything with aspnet/MessagePack-CSharp, it's unrelated to SignalR's usage of MessagePack.

because i saw this https://github.com/aspnet/AspNetCore/blob/release/3.1/.gitmodules#L7
but if i understand correctly, the submodule is only used for building the sources inside Blazor itself (for the custom BlazorHubProtocol)
I saw the protocol, but it was not affected by the breaking change either ;)

Now i understand why i saw both the submodule, and the PackageReference
thx a lot i was not sure if they needed to be aligned or not

@tebeco tebeco changed the base branch from release/3.1 to master January 5, 2020 19:17
@tebeco
Copy link
Contributor Author

tebeco commented Jan 5, 2020

@BrennanConroy

  • dropped submodule change
  • dropped Blazor related csproj changes
  • re-word few commits
  • rebased / pushed / force-with-lease on master

@Tratcher Tratcher removed their request for review January 5, 2020 19:44
@tebeco
Copy link
Contributor Author

tebeco commented Jan 5, 2020

Not sure how should i proceed for all the // REVIEW that i added,
i would love a feedback on these ;)

Also i'm not happy with 2 of them in RedisProtocol to be honest

@BrennanConroy BrennanConroy added the area-signalr Includes: SignalR clients and servers label Jan 5, 2020
@AArnott
Copy link
Contributor

AArnott commented Jan 6, 2020

I'm glad we're all on the same page that the submodule is irrelevant here. But just to respond to this question:

(not sure why the double fork here ^^)

The fork relationship through aarnott first and then to neuecc is just a historical artifact from when the AArnott account had the 2.0 source code before neuecc had it. So it was logical to fork from the newer repo. Now that the fork is created, the relationship of being a direct child or grand-child of neuecc is irrelevant AFAIK because github doesn't do anything for you whether you're an immediate vs. distant fork 'relative'. You still have to pull from some upstream and push to your own repo to stay current. And ya, the aspnet fork of messagepack probably hasn't pushed the 2.0 RTM release of messagepack from neuecc yet as they haven't needed to but may choose to do that in the future.

But again, irrelevant to what you're doing, so that's great.

@AArnott
Copy link
Contributor

AArnott commented Jan 6, 2020

if it could be part of 3.1, i'm not sure if it's concidered as a Breaking change

It is a breaking change to update non-patch versions of dependencies.

Given that (which makes total sense), perhaps the SignalR 3.1 change should be to add the upper-bound requirement on the messagepack reference. This way the expected behavior of someone who references both ASP.NET Core and MessagePack directly will get a nuget error to tell them that SignalR is incompatible with MessagePack 2.0.
That would be a much smaller and simpler PR and IMO more likely to be accepted.

Then we could start a more long-running PR to upgrade SignalR to consume MessagePack 2.0 which would complete for some major upgrade of SignalR.

Copy link
Member

@BrennanConroy BrennanConroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking forward to perf numbers once this is working

@tebeco
Copy link
Contributor Author

tebeco commented Jan 6, 2020

Any idea if the Pull Request Labeler / triage error is on my side ?
I see some HttpError from node and YARN_TELEMETRY errors.

I'm not sure where to look at, it feels like something is missing somewhere, bad url or service being down

@tebeco tebeco requested review from BrennanConroy and removed request for a team January 6, 2020 08:17
@BrennanConroy BrennanConroy removed the request for review from SteveSandersonMS January 6, 2020 16:04
@tebeco
Copy link
Contributor Author

tebeco commented Jan 14, 2020

Generally we click "Squash and merge" when merging a PR, but if you'd like to manually do that and customize the commit message go for it :)

well then "squash and merge", i try to give git history a sense for reviewer so people can navigate through it step by step, then review commits
giving the proportion of this repo i understand the "squash them all" idea
i'll go with me being lazy and let you click that ;)

@AArnott
Copy link
Contributor

AArnott commented Jan 14, 2020

The BannedApiAnalyzers sounds interesting, but lets not complicate this PR with it.

Don't use it. We have a dedicated analyzer that does a much more thorough job of keeping you on the happy path in MessagePack-CSharp/MessagePack-CSharp#769 that will be available very soon.

@dougbu
Copy link
Contributor

dougbu commented Jan 14, 2020

do you have any clean way for contributor to re-run the CI

Best way is to reach out to anyone on the team, asking them to do the /azp run thing. @dotnet/aspnet-build can help if you don't already know @BrennanConroy 😺

@tebeco
Copy link
Contributor Author

tebeco commented Jan 15, 2020

I don't think we care about caching CompositeResolver as this class should only be created once for the lifetime of the app.

sounds good to me i did not took shoot yet, i was not sure about all lifetime duration. I had doubt especially since the IHubContext is either Transient/Scoped and i did not took time yet to see how IHubProtocol was both Registered and Resolved (hence the comment, totally agree on keeping this PR to be "small" first, then maybe other PR is possible enhancement)

The BannedApiAnalyzers sounds interesting, but lets not complicate this PR with it.

@BrennanConroy dont worry i took a look at it elsewhere ( #18291)

Don't use it. We have a dedicated analyzer that does a much more thorough job

@AArnott should i close #183291 then

Best way is to reach out to anyone on the team, asking them to do the /azp run thing. @dotnet/aspnet-build can help if you don't already know @BrennanConroy 😺

@dougbu
you just volunteered as a tribute right ? my '@' key is ready 👻

@tebeco
Copy link
Contributor Author

tebeco commented Jan 16, 2020

Early run =>

  • the bench result are hard to compare for now
  • only ran on this branch

Soon, i'll reset on master / Rebuild / Rerun (to get the same base code but with MessagePack v1.7.x
I'll probably manually Edit the MD file export, and only let the line like MessagePack v1 and MessagePack v2

dotnet run -c Release ==> Selected Benchmarks : 9 12 for HubProtocolBenchmark and RedisProtocolBenchmark

HubProtocolBenchmark (MessagePack v2)

BenchmarkDotNet=v0.10.13, OS=Windows 10.0.19041
Intel Core i9-9980HK CPU 2.40GHz, 1 CPU, 16 logical cores and 8 physical cores
.NET Core SDK=5.0.100-alpha1-015752
  [Host]     : .NET Core 5.0.0-alpha.1.19562.8 (CoreCLR 5.0.19.56201, CoreFX 5.0.19.55607), 64bit RyuJIT
  Job-VDDHMF : .NET Core 5.0.0-alpha.1.19562.8 (CoreCLR 5.0.19.56201, CoreFX 5.0.19.55607), 64bit RyuJIT

Runtime=Core  Server=True  Toolchain=.NET Core 5.0  
RunStrategy=Throughput  
Method Input HubProtocol Mean Error StdDev Op/s Gen 0 Allocated
ReadSingleMessage FewArguments Json 1,408.0 ns 27.445 ns 37.567 ns 710,233.4 - 224 B
WriteSingleMessage FewArguments Json 677.0 ns 12.220 ns 11.431 ns 1,477,014.5 - 80 B
ReadSingleMessage FewArguments MsgPack 508.7 ns 8.661 ns 8.101 ns 1,965,800.8 0.0010 224 B
WriteSingleMessage FewArguments MsgPack 299.6 ns 4.098 ns 3.633 ns 3,337,701.3 - 48 B
ReadSingleMessage FewArguments NewtonsoftJson 1,473.4 ns 7.938 ns 7.037 ns 678,724.0 0.0057 960 B
WriteSingleMessage FewArguments NewtonsoftJson 1,237.1 ns 18.995 ns 17.767 ns 808,353.6 0.0038 784 B
ReadSingleMessage LargeArguments Json 7,747.9 ns 40.263 ns 35.692 ns 129,067.9 0.2136 30904 B
WriteSingleMessage LargeArguments Json 12,189.6 ns 115.550 ns 102.432 ns 82,037.1 0.1526 23976 B
ReadSingleMessage LargeArguments MsgPack 3,090.9 ns 17.733 ns 15.720 ns 323,529.3 0.2136 30904 B
WriteSingleMessage LargeArguments MsgPack 2,547.6 ns 48.178 ns 45.066 ns 392,530.1 0.1373 20528 B
ReadSingleMessage LargeArguments NewtonsoftJson 54,095.9 ns 435.854 ns 407.698 ns 18,485.7 0.3662 58840 B
WriteSingleMessage LargeArguments NewtonsoftJson 30,820.9 ns 248.092 ns 232.065 ns 32,445.5 0.1221 24768 B
ReadSingleMessage ManyArguments Json 2,879.0 ns 40.406 ns 37.796 ns 347,348.8 - 424 B
WriteSingleMessage ManyArguments Json 1,212.3 ns 20.346 ns 19.032 ns 824,883.1 - 120 B
ReadSingleMessage ManyArguments MsgPack 927.8 ns 14.998 ns 12.524 ns 1,077,849.4 0.0019 400 B
WriteSingleMessage ManyArguments MsgPack 502.5 ns 6.243 ns 5.840 ns 1,990,093.7 - 72 B
ReadSingleMessage ManyArguments NewtonsoftJson 2,685.6 ns 17.956 ns 14.994 ns 372,353.9 0.0076 1504 B
WriteSingleMessage ManyArguments NewtonsoftJson 2,390.1 ns 20.629 ns 19.296 ns 418,398.3 0.0114 1992 B
ReadSingleMessage NoArguments Json 505.3 ns 7.987 ns 7.471 ns 1,978,974.2 - 96 B
WriteSingleMessage NoArguments Json 231.9 ns 3.596 ns 3.364 ns 4,312,739.9 0.0005 72 B
ReadSingleMessage NoArguments MsgPack 295.1 ns 1.561 ns 1.219 ns 3,388,182.0 0.0005 120 B
WriteSingleMessage NoArguments MsgPack 156.9 ns 3.005 ns 2.663 ns 6,372,944.3 0.0002 40 B
ReadSingleMessage NoArguments NewtonsoftJson 674.8 ns 5.865 ns 5.487 ns 1,481,944.1 0.0038 584 B
WriteSingleMessage NoArguments NewtonsoftJson 583.0 ns 3.126 ns 2.611 ns 1,715,365.5 0.0019 384 B

RedisProtocolBenchmark (MessagePack v2)

BenchmarkDotNet=v0.10.13, OS=Windows 10.0.19041
Intel Core i9-9980HK CPU 2.40GHz, 1 CPU, 16 logical cores and 8 physical cores
.NET Core SDK=5.0.100-alpha1-015752
  [Host]     : .NET Core 5.0.0-alpha.1.19562.8 (CoreCLR 5.0.19.56201, CoreFX 5.0.19.55607), 64bit RyuJIT
  Job-VDDHMF : .NET Core 5.0.0-alpha.1.19562.8 (CoreCLR 5.0.19.56201, CoreFX 5.0.19.55607), 64bit RyuJIT

Runtime=Core  Server=True  Toolchain=.NET Core 5.0  
RunStrategy=Throughput  
Method Mean Error StdDev Op/s Gen 0 Allocated
WriteAck 99.53 ns 1.0613 ns 0.9928 ns 10,047,286.1 0.0001 32 B
WriteGroupCommand 156.25 ns 1.7986 ns 1.6825 ns 6,400,086.4 0.0002 56 B
WriteInvocationNoExclusions 492.51 ns 4.8058 ns 4.4954 ns 2,030,432.1 0.0019 344 B
WriteInvocationSmallExclusions 569.81 ns 4.0341 ns 3.7735 ns 1,754,966.5 0.0029 448 B
WriteInvocationLargeExclusions 1,024.88 ns 7.7735 ns 7.2714 ns 975,720.9 0.0057 1056 B
ReadAck 44.09 ns 0.8997 ns 1.8780 ns 22,681,088.2 - 0 B
ReadGroupCommand 170.60 ns 3.3896 ns 6.1121 ns 5,861,596.0 0.0007 120 B
ReadInvocationNoExclusions 472.52 ns 9.2724 ns 11.7266 ns 2,116,300.1 0.0019 328 B
ReadInvocationSmallExclusions 520.59 ns 11.5869 ns 15.4682 ns 1,920,882.2 0.0029 544 B
ReadInvocationLargeExclusions 1,272.12 ns 7.4166 ns 6.9375 ns 786,087.4 0.0153 2272 B

@tebeco
Copy link
Contributor Author

tebeco commented Jan 16, 2020

@BrennanConroy

BenchmarkDotNet=v0.10.13, OS=Windows 10.0.19041
Intel Core i9-9980HK CPU 2.40GHz, 1 CPU, 16 logical cores and 8 physical cores
.NET Core SDK=5.0.100-alpha1-015752
  [Host]     : .NET Core 5.0.0-alpha.1.19562.8 (CoreCLR 5.0.19.56201, CoreFX 5.0.19.55607), 64bit RyuJIT
  Job-MRPWQU : .NET Core 5.0.0-alpha.1.19562.8 (CoreCLR 5.0.19.56201, CoreFX 5.0.19.55607), 64bit RyuJIT

Runtime=Core  Server=True  Toolchain=.NET Core 5.0  
RunStrategy=Throughput  

HubProtocolBenchmark (MessagePack v1 versus v2)

I'm very very suprised about what i see :

  • Slower for the Read
  • Faster for the Write

(i'll tree to re-run both to be sure of that diff)

Method Input HubProtocol Mean Error StdDev Median Op/s Gen 0 Allocated
ReadSingleMessage FewArguments MsgPack v1 398.4 ns 5.248 ns 4.652 ns 398.5 ns 2,509,937.6 0.0014 224 B
ReadSingleMessage FewArguments MsgPack v2 508.7 ns 8.661 ns 8.101 ns 1,965,800.8 0.0010 224 B
WriteSingleMessage FewArguments MsgPack v1 396.1 ns 1.972 ns 1.540 ns 396.1 ns 2,524,851.3 - 48 B
WriteSingleMessage FewArguments MsgPack v2 299.6 ns 4.098 ns 3.633 ns 3,337,701.3 - 48 B
ReadSingleMessage LargeArguments MsgPack v1 2,934.5 ns 26.977 ns 25.235 ns 2,923.3 ns 340,778.7 0.2136 30904 B
ReadSingleMessage LargeArguments MsgPack v2 3,090.9 ns 17.733 ns 15.720 ns 323,529.3 0.2136 30904 B
WriteSingleMessage LargeArguments MsgPack v1 3,490.1 ns 68.844 ns 139.068 ns 3,418.6 ns 286,524.7 0.1373 20528 B
WriteSingleMessage LargeArguments MsgPack v2 2,547.6 ns 48.178 ns 45.066 ns 392,530.1 0.1373 20528 B
ReadSingleMessage ManyArguments MsgPack v1 711.8 ns 7.832 ns 7.326 ns 713.1 ns 1,404,965.4 0.0019 400 B
ReadSingleMessage ManyArguments MsgPack v2 927.8 ns 14.998 ns 12.524 ns 1,077,849.4 0.0019 400 B
WriteSingleMessage ManyArguments MsgPack v1 730.3 ns 14.377 ns 15.383 ns 721.2 ns 1,369,321.7 - 72 B
WriteSingleMessage ManyArguments MsgPack v2 502.5 ns 6.243 ns 5.840 ns 1,990,093.7 - 72 B
ReadSingleMessage NoArguments MsgPack v1 213.4 ns 1.640 ns 1.534 ns 213.4 ns 4,685,100.4 0.0007 120 B
ReadSingleMessage NoArguments MsgPack v2 295.1 ns 1.561 ns 1.219 ns 3,388,182.0 0.0005 120 B
WriteSingleMessage NoArguments MsgPack v1 233.2 ns 2.932 ns 2.743 ns 233.2 ns 4,288,751.0 - 40 B
WriteSingleMessage NoArguments MsgPack v2 156.9 ns 3.005 ns 2.663 ns 6,372,944.3 0.0002 40 B

RedisProtocolBenchmark (MessagePack v1 versus v2)

On the good side RedisProtocolBenchmark seems to benefits for both Read and Write

Method Mean Error StdDev Op/s Gen 0 Allocated
WriteAck v1 107.34 ns 2.1550 ns 5.2862 ns 9,316,580.2 0.0001 32 B
WriteAck v2 99.53 ns 1.0613 ns 0.9928 ns 10,047,286.1 0.0001 32 B
WriteGroupCommand v1 249.21 ns 4.9007 ns 6.5423 ns 4,012,617.5 - 56 B
WriteGroupCommand v2 156.25 ns 1.7986 ns 1.6825 ns 6,400,086.4 0.0002 56 B
WriteInvocationNoExclusions v1 613.08 ns 11.8506 ns 11.0850 ns 1,631,119.5 0.0019 344 B
WriteInvocationNoExclusions v2 492.51 ns 4.8058 ns 4.4954 ns 2,030,432.1 0.0019 344 B
WriteInvocationSmallExclusions v1 742.64 ns 10.8510 ns 10.1500 ns 1,346,547.1 0.0029 448 B
WriteInvocationSmallExclusions v2 569.81 ns 4.0341 ns 3.7735 ns 1,754,966.5 0.0029 448 B
WriteInvocationLargeExclusions v1 1,621.54 ns 7.2388 ns 6.0447 ns 616,699.6 0.0057 1056 B
WriteInvocationLargeExclusions v2 1,024.88 ns 7.7735 ns 7.2714 ns 975,720.9 0.0057 1056 B
ReadAck v1 33.50 ns 0.7159 ns 0.6696 ns 29,855,141.2 - 0 B
ReadAck v2 44.09 ns 0.8997 ns 1.8780 ns 22,681,088.2 - 0 B
ReadGroupCommand v1 177.50 ns 1.9975 ns 1.8685 ns 5,633,755.4 0.0007 120 B
ReadGroupCommand v2 170.60 ns 3.3896 ns 6.1121 ns 5,861,596.0 0.0007 120 B
ReadInvocationNoExclusions v1 278.72 ns 2.1459 ns 2.0073 ns 3,587,810.3 0.0019 328 B
ReadInvocationNoExclusions v2 472.52 ns 9.2724 ns 11.7266 ns 2,116,300.1 0.0019 328 B
ReadInvocationSmallExclusions v1 384.90 ns 2.0552 ns 1.9224 ns 2,598,075.3 0.0033 544 B
ReadInvocationSmallExclusions v2 520.59 ns 11.5869 ns 15.4682 ns 1,920,882.2 0.0029 544 B
ReadInvocationLargeExclusions v1 1,280.25 ns 10.8676 ns 10.1656 ns 781,094.4 0.0153 2272 B
ReadInvocationLargeExclusions v2 1,272.12 ns 7.4166 ns 6.9375 ns 786,087.4 0.0153 2272 B

@AArnott
Copy link
Contributor

AArnott commented Jan 16, 2020

@tebeco Writing is faster because we don't have to allocate and copy memory. Especially when writing large messages, v1 would resize arrays repeatedly which got progressively more expensive each time it did it. With v2 we can write to non-contiguous buffers so we just ask for more memory when we need it, and can write directly into the memory offered by an existing IBufferWriter.

Reading is a bit slower because we no longer demand reading from a byte[]. Nothing is faster than reading from byte[]. But now we read from Memory<byte> or ReadOnlySequence<byte>, both of which are a little slower as it adds abstractions.
But this is in micro-benchmarking. We win it back (sometimes at least) in real world scenarios where the message to be deserialized may be large such that you don't want to require contiguous memory to allocate it.

@tebeco
Copy link
Contributor Author

tebeco commented Jan 16, 2020

Seems pretty logic explained like this, i just did not excepted that amount of difference to be honest

@AArnott
Copy link
Contributor

AArnott commented Jan 16, 2020

It depends on what you're measuring. It's 2X slower or more when you're deserializing an integer. But the cost amortizes out as the size of the message increases. By the time you're deserializing a type with several members in it, the difference becomes insignificant.

@tebeco
Copy link
Contributor Author

tebeco commented Jan 16, 2020

As i said i would re-run, and not to pollute upper table which are already big enough :
For any one curious, please read #18133 (comment) and #18133 (comment)

v1 Microbenchmarks were run on commit 6dd3d6ec49d7e61a71cc825dcbdc9533d2620d7d
v2 Microbenchmarks were run on commit 50f236866127f56de51263491dd55f018a00fae6

For both run 3, i edited HubProtocolBenchmark.cs
from [Params(Protocol.MsgPack, Protocol.Json, Protocol.NewtonsoftJson)]
to [Params(Protocol.MsgPack)]

Method Input HubProtocol Mean Error StdDev Median Op/s Gen 0 Gen 1 Allocated
ReadSingleMessage FewArguments MsgPack v1 (run 1) 398.4 ns 5.248 ns 4.652 ns 398.5 ns 2,509,937.6 0.0014 - 224 B
ReadSingleMessage FewArguments MsgPack v1 (run 2) 405.2 ns 4.284 ns 4.007 ns 2,467,797.6 0.0014 - 224 B
ReadSingleMessage FewArguments MsgPack v1 (run 3) 421.2 ns 7.357 ns 6.882 ns 2,374,304.7 0.0014 - 224 B
ReadSingleMessage FewArguments MsgPack v2 (run 1) 508.7 ns 8.661 ns 8.101 ns 1,965,800.8 0.0010 - 224 B
ReadSingleMessage FewArguments MsgPack v2 (run 2) 512.7 ns 4.427 ns 3.925 ns 1,950,393.1 0.0010 - 224 B
ReadSingleMessage FewArguments MsgPack v2 (run 3) 525.9 ns 9.853 ns 10.542 ns 1,901,653.6 0.0010 - 224 B
WriteSingleMessage FewArguments MsgPack v1 (run 1) 396.1 ns 1.972 ns 1.540 ns 396.1 ns 2,524,851.3 - - 48 B
WriteSingleMessage FewArguments MsgPack v1 (run 2) 413.8 ns 8.131 ns 9.985 ns 2,416,627.3 - - 48 B
WriteSingleMessage FewArguments MsgPack v1 (run 3) 405.4 ns 9.780 ns 12.368 ns 2,466,956.8 - - 48 B
WriteSingleMessage FewArguments MsgPack v2 (run 1) 299.6 ns 4.098 ns 3.633 ns 3,337,701.3 - - 48 B
WriteSingleMessage FewArguments MsgPack v2 (run 2) 294.2 ns 5.693 ns 5.847 ns 3,398,619.2 - - 48 B
WriteSingleMessage FewArguments MsgPack v2 (run 3) 309.8 ns 5.982 ns 6.649 ns 3,227,630.1 - - 48 B
ReadSingleMessage LargeArguments MsgPack v1 (run 1) 2,934.5 ns 26.977 ns 25.235 ns 2,923.3 ns 340,778.7 0.2136 - 30904 B
ReadSingleMessage LargeArguments MsgPack v1 (run 2) 2,940.6 ns 23.967 ns 22.419 ns 340,063.5 0.2136 0.0038 30904 B
ReadSingleMessage LargeArguments MsgPack v1 (run 3) 2,971.4 ns 14.375 ns 12.004 ns 336,546.5 0.2136 - 30904 B
ReadSingleMessage LargeArguments MsgPack v2 (run 1) 3,090.9 ns 17.733 ns 15.720 ns 323,529.3 0.2136 - 30904 B
ReadSingleMessage LargeArguments MsgPack v2 (run 2) 3,074.4 ns 22.390 ns 19.848 ns 325,264.8 0.2136 - 30904 B
ReadSingleMessage LargeArguments MsgPack v2 (run 3) 3,060.0 ns 10.783 ns 10.086 ns 326,796.1 0.2136 - 30904 B
WriteSingleMessage LargeArguments MsgPack v1 (run 1) 3,490.1 ns 68.844 ns 139.068 ns 3,418.6 ns 286,524.7 0.1373 - 20528 B
WriteSingleMessage LargeArguments MsgPack v1 (run 2) 3,423.2 ns 20.921 ns 18.546 ns 292,128.4 0.1373 - 20528 B
WriteSingleMessage LargeArguments MsgPack v1 (run 3) 3,353.7 ns 21.173 ns 19.805 ns 298,177.0 0.1373 - 20528 B
WriteSingleMessage LargeArguments MsgPack v2 (run 1) 2,547.6 ns 48.178 ns 45.066 ns 392,530.1 0.1373 - 20528 B
WriteSingleMessage LargeArguments MsgPack v2 (run 2) 2,510.5 ns 24.158 ns 22.598 ns 398,328.7 0.1373 - 20528 B
WriteSingleMessage LargeArguments MsgPack v2 (run 3) 2,509.0 ns 10.647 ns 8.891 ns 398,567.7 0.1373 - 20528 B
ReadSingleMessage ManyArguments MsgPack v1 (run 1) 711.8 ns 7.832 ns 7.326 ns 713.1 ns 1,404,965.4 0.0019 - 400 B
ReadSingleMessage ManyArguments MsgPack v1 (run 2) 681.4 ns 6.059 ns 5.668 ns 1,467,518.8 0.0019 - 400 B
ReadSingleMessage ManyArguments MsgPack v1 (run 3) 715.2 ns 14.944 ns 21.433 ns 1,398,146.5 0.0019 - 400 B
ReadSingleMessage ManyArguments MsgPack v2 (run 1) 927.8 ns 14.998 ns 12.524 ns 1,077,849.4 0.0019 - 400 B
ReadSingleMessage ManyArguments MsgPack v2 (run 2) 925.6 ns 12.070 ns 10.699 ns 1,080,423.4 0.0019 - 400 B
ReadSingleMessage ManyArguments MsgPack v2 (run 3) 919.7 ns 8.503 ns 7.954 ns 1,087,304.8 0.0019 - 400 B
WriteSingleMessage ManyArguments MsgPack v1 (run 1) 730.3 ns 14.377 ns 15.383 ns 721.2 ns 1,369,321.7 - - 72 B
WriteSingleMessage ManyArguments MsgPack v1 (run 2) 729.1 ns 11.961 ns 11.189 ns 1,371,515.7 - - 72 B
WriteSingleMessage ManyArguments MsgPack v1 (run 3) 731.5 ns 12.487 ns 11.680 ns 1,367,144.6 - - 72 B
WriteSingleMessage ManyArguments MsgPack v2 (run 1) 502.5 ns 6.243 ns 5.840 ns 1,990,093.7 - - 72 B
WriteSingleMessage ManyArguments MsgPack v2 (run 2) 518.8 ns 7.888 ns 6.993 ns 1,927,563.6 - - 72 B
WriteSingleMessage ManyArguments MsgPack v2 (run 3) 518.0 ns 2.311 ns 2.048 ns 1,930,427.3 - - 72 B
ReadSingleMessage NoArguments MsgPack v1 (run 1) 213.4 ns 1.640 ns 1.534 ns 213.4 ns 4,685,100.4 0.0007 - 120 B
ReadSingleMessage NoArguments MsgPack v1 (run 2) 215.1 ns 2.276 ns 2.129 ns 4,649,272.4 0.0007 - 120 B
ReadSingleMessage NoArguments MsgPack v1 (run 3) 211.1 ns 1.265 ns 1.183 ns 4,736,005.3 0.0007 - 120 B
ReadSingleMessage NoArguments MsgPack v2 (run 1) 295.1 ns 1.561 ns 1.219 ns 3,388,182.0 0.0005 - 120 B
ReadSingleMessage NoArguments MsgPack v2 (run 2) 297.4 ns 2.934 ns 2.601 ns 3,362,734.2 0.0005 - 120 B
ReadSingleMessage NoArguments MsgPack v2 (run 3) 297.6 ns 3.277 ns 3.065 ns 3,360,338.0 0.0005 - 120 B
WriteSingleMessage NoArguments MsgPack v1 (run 1) 233.2 ns 2.932 ns 2.743 ns 233.2 ns 4,288,751.0 - - 40 B
WriteSingleMessage NoArguments MsgPack v1 (run 2) 235.7 ns 4.637 ns 5.340 ns 4,242,480.5 0.0002 - 40 B
WriteSingleMessage NoArguments MsgPack v1 (run 3) 230.7 ns 4.077 ns 3.813 ns 4,333,982.1 0.0002 - 40 B
WriteSingleMessage NoArguments MsgPack v2 (run 1) 156.9 ns 3.005 ns 2.663 ns 6,372,944.3 0.0002 - 40 B
WriteSingleMessage NoArguments MsgPack v2 (run 2) 163.9 ns 2.220 ns 2.077 ns 6,101,538.8 0.0002 - 40 B
WriteSingleMessage NoArguments MsgPack v2 (run 3) 159.3 ns 2.506 ns 2.344 ns 6,278,890.3 0.0002 - 40 B

@tebeco
Copy link
Contributor Author

tebeco commented Jan 20, 2020

@BrennanConroy
Anything else in this missing PR ?

I can rebase on master and also update MessagePack from 2.0.323 to 2.0.335

@BrennanConroy
Copy link
Member

Anything else in this missing PR ?

Sorry, I was busy dealing with some other things. I'll take a look at this right now.

I can rebase on master and also update MessagePack from 2.0.323 to 2.0.335

No need to rebase on master, no one is touching these files so there should be any issues :)

Updating to the new version would be nice though!

@tebeco
Copy link
Contributor Author

tebeco commented Jan 21, 2020

i'll update the version of messagepack tomorrow in the morning (10PM here)

@tebeco
Copy link
Contributor Author

tebeco commented Jan 21, 2020

and i'll ping @dougbu to run the build because he volunteered (even if the trigger worked :trollface: )

@dougbu
Copy link
Contributor

dougbu commented Jan 22, 2020

i'll ping @dougbu to run the build

Shouldn't be necessary but if it is sure. (Updating the branch should cause the builds to run.)

@BrennanConroy
Copy link
Member

Thanks for the great contribution @tebeco !

@tebeco
Copy link
Contributor Author

tebeco commented Jan 25, 2020

thx everyone for the patience and various form of help.

@davidfowl
Copy link
Member

This is great to see! Those reading numbers look at bit worse but I'd expect that because of what @AArnott said. Hopefully it doesn't matter for larger object graphs

@tebeco tebeco deleted the attempting-to-update-msgpack branch March 17, 2020 14:53
@tebeco
Copy link
Contributor Author

tebeco commented Mar 17, 2020

@anurse
@BrennanConroy

As this PR does not have any Issue related, it does not appear in the release note of the 5.0-preview1
It's my fault creating a PR without related issue.

Not sure if it could / should be changed somehow
I tried to check other PR labelled after 5.0-preview1 without linked issue ... but maybe you already have scanning code to detect that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-signalr Includes: SignalR clients and servers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants