Skip to content

refactor: replace 3rd-party xxhash impl with in-house reimpl #2310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions com.unity.netcode.gameobjects/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

# Changelog

All notable changes to this project will be documented in this file.
Expand All @@ -10,11 +9,13 @@ Additional documentation and release notes are available at [Multiplayer Documen
## [Unreleased]

### Added

- Added `NetworkObject` auto-add helper and Multiplayer Tools install reminder settings to Project Settings. (#2285)
- Added `public string DisconnectReason` getter to `NetworkManager` and `string Reason` to `ConnectionApprovalResponse`. Allows connection approval to communicate back a reason. Also added `public void DisconnectClient(ulong clientId, string reason)` allowing setting a disconnection reason, when explicitly disconnecting a client.

### Changed

- Changed 3rd-party `XXHash` (32 & 64) implementation with an in-house reimplementation (#2310)
- Optimized bandwidth usage by encoding most integer fields using variable-length encoding. (#2276)

### Fixed
Expand All @@ -25,9 +26,10 @@ Additional documentation and release notes are available at [Multiplayer Documen
- Fixed issue where in-scene placed `NetworkObjects` were not honoring the `AutoObjectParentSync` property. (#2281)
- Fixed the issue where `NetworkManager.OnClientConnectedCallback` was being invoked before in-scene placed `NetworkObject`s had been spawned when starting `NetworkManager` as a host. (#2277)
- Creating a `FastBufferReader` with `Allocator.None` will not result in extra memory being allocated for the buffer (since it's owned externally in that scenario). (#2265)

### Removed
- Removed the `NetworkObject` auto-add and Multiplayer Tools install reminder settings from the Menu interface. (#2285)

- Removed the `NetworkObject` auto-add and Multiplayer Tools install reminder settings from the Menu interface. (#2285)

## [1.1.0] - 2022-10-21

Expand Down Expand Up @@ -179,6 +181,7 @@ Additional documentation and release notes are available at [Multiplayer Documen
- Removed `ClientNetworkTransform` from the package samples and moved to Boss Room's Utilities package which can be found [here](https://github.com/Unity-Technologies/com.unity.multiplayer.samples.coop/blob/main/Packages/com.unity.multiplayer.samples.coop/Utilities/Net/ClientAuthority/ClientNetworkTransform.cs) (#1912)

### Fixed

- Fixed issue where `NetworkSceneManager` did not synchronize despawned in-scene placed NetworkObjects. (#1898)
- Fixed `NetworkTransform` generating false positive rotation delta checks when rolling over between 0 and 360 degrees. (#1890)
- Fixed client throwing an exception if it has messages in the outbound queue when processing the `NetworkEvent.Disconnect` event and is using UTP. (#1884)
Expand Down Expand Up @@ -232,10 +235,12 @@ Additional documentation and release notes are available at [Multiplayer Documen
## [1.0.0-pre.6] - 2022-03-02

### Added

- NetworkAnimator now properly synchrhonizes all animation layers as well as runtime-adjusted weighting between them (#1765)
- Added first set of tests for NetworkAnimator - parameter syncing, trigger set / reset, override network animator (#1735)

### Fixed

- Fixed an issue where sometimes the first client to connect to the server could see messages from the server as coming from itself. (#1683)
- Fixed an issue where clients seemed to be able to send messages to ClientId 1, but these messages would actually still go to the server (id 0) instead of that client. (#1683)
- Improved clarity of error messaging when a client attempts to send a message to a destination other than the server, which isn't allowed. (#1683)
Expand Down Expand Up @@ -287,6 +292,7 @@ Additional documentation and release notes are available at [Multiplayer Documen
- Removed `FixedQueue`, `StreamExtensions`, `TypeExtensions` (#1398)

### Fixed

- Fixed in-scene NetworkObjects that are moved into the DDOL scene not getting restored to their original active state (enabled/disabled) after a full scene transition (#1354)
- Fixed invalid IL code being generated when using `this` instead of `this ref` for the FastBufferReader/FastBufferWriter parameter of an extension method. (#1393)
- Fixed an issue where if you are running as a server (not host) the LoadEventCompleted and UnloadEventCompleted events would fire early by the NetworkSceneManager (#1379)
Expand All @@ -301,6 +307,7 @@ Additional documentation and release notes are available at [Multiplayer Documen
- Fixed network tick value sometimes being duplicated or skipped. (#1614)

### Changed

- The SDK no longer limits message size to 64k. (The transport may still impose its own limits, but the SDK no longer does.) (#1384)
- Updated com.unity.collections to 1.1.0 (#1451)
- NetworkManager's GameObject is no longer allowed to be nested under one or more GameObject(s).(#1484)
Expand Down
248 changes: 248 additions & 0 deletions com.unity.netcode.gameobjects/Runtime/Hashing/XXHash.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,248 @@
using System;
using System.Text;
using System.Runtime.CompilerServices;

namespace Unity.Netcode
{
internal static class XXHash
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe uint Hash32(byte* input, int length, uint seed = 0)
{
unchecked
{
const uint prime1 = 2654435761u;
const uint prime2 = 2246822519u;
const uint prime3 = 3266489917u;
const uint prime4 = 0668265263u;
const uint prime5 = 0374761393u;

uint hash = seed + prime5;

if (length >= 16)
{
uint val0 = seed + prime1 + prime2;
uint val1 = seed + prime2;
uint val2 = seed + 0;
uint val3 = seed - prime1;

int count = length >> 4;
for (int i = 0; i < count; i++)
{
var pos0 = *(uint*)(input + 0);
var pos1 = *(uint*)(input + 4);
var pos2 = *(uint*)(input + 8);
var pos3 = *(uint*)(input + 12);

val0 += pos0 * prime2;
val0 = (val0 << 13) | (val0 >> (32 - 13));
val0 *= prime1;

val1 += pos1 * prime2;
val1 = (val1 << 13) | (val1 >> (32 - 13));
val1 *= prime1;

val2 += pos2 * prime2;
val2 = (val2 << 13) | (val2 >> (32 - 13));
val2 *= prime1;

val3 += pos3 * prime2;
val3 = (val3 << 13) | (val3 >> (32 - 13));
val3 *= prime1;

input += 16;
}

hash = ((val0 << 01) | (val0 >> (32 - 01))) +
((val1 << 07) | (val1 >> (32 - 07))) +
((val2 << 12) | (val2 >> (32 - 12))) +
((val3 << 18) | (val3 >> (32 - 18)));
}

hash += (uint)length;

length &= 15;
while (length >= 4)
{
hash += *(uint*)input * prime3;
hash = ((hash << 17) | (hash >> (32 - 17))) * prime4;
input += 4;
length -= 4;
}
while (length > 0)
{
hash += *input * prime5;
hash = ((hash << 11) | (hash >> (32 - 11))) * prime1;
++input;
--length;
}

hash ^= hash >> 15;
hash *= prime2;
hash ^= hash >> 13;
hash *= prime3;
hash ^= hash >> 16;

return hash;
}
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static unsafe ulong Hash64(byte* input, int length, uint seed = 0)
{
unchecked
{
const ulong prime1 = 11400714785074694791ul;
const ulong prime2 = 14029467366897019727ul;
const ulong prime3 = 01609587929392839161ul;
const ulong prime4 = 09650029242287828579ul;
const ulong prime5 = 02870177450012600261ul;

ulong hash = seed + prime5;

if (length >= 32)
{
ulong val0 = seed + prime1 + prime2;
ulong val1 = seed + prime2;
ulong val2 = seed + 0;
ulong val3 = seed - prime1;

int count = length >> 5;
for (int i = 0; i < count; i++)
{
var pos0 = *(ulong*)(input + 0);
var pos1 = *(ulong*)(input + 8);
var pos2 = *(ulong*)(input + 16);
var pos3 = *(ulong*)(input + 24);

val0 += pos0 * prime2;
val0 = (val0 << 31) | (val0 >> (64 - 31));
val0 *= prime1;

val1 += pos1 * prime2;
val1 = (val1 << 31) | (val1 >> (64 - 31));
val1 *= prime1;

val2 += pos2 * prime2;
val2 = (val2 << 31) | (val2 >> (64 - 31));
val2 *= prime1;

val3 += pos3 * prime2;
val3 = (val3 << 31) | (val3 >> (64 - 31));
val3 *= prime1;

input += 32;
}

hash = ((val0 << 01) | (val0 >> (64 - 01))) +
((val1 << 07) | (val1 >> (64 - 07))) +
((val2 << 12) | (val2 >> (64 - 12))) +
((val3 << 18) | (val3 >> (64 - 18)));

val0 *= prime2;
val0 = (val0 << 31) | (val0 >> (64 - 31));
val0 *= prime1;
hash ^= val0;
hash = hash * prime1 + prime4;

val1 *= prime2;
val1 = (val1 << 31) | (val1 >> (64 - 31));
val1 *= prime1;
hash ^= val1;
hash = hash * prime1 + prime4;

val2 *= prime2;
val2 = (val2 << 31) | (val2 >> (64 - 31));
val2 *= prime1;
hash ^= val2;
hash = hash * prime1 + prime4;

val3 *= prime2;
val3 = (val3 << 31) | (val3 >> (64 - 31));
val3 *= prime1;
hash ^= val3;
hash = hash * prime1 + prime4;
}

hash += (ulong)length;

length &= 31;
while (length >= 8)
{
ulong lane = *(ulong*)input * prime2;
lane = ((lane << 31) | (lane >> (64 - 31))) * prime1;
hash ^= lane;
hash = ((hash << 27) | (hash >> (64 - 27))) * prime1 + prime4;
input += 8;
length -= 8;
}
if (length >= 4)
{
hash ^= *(uint*)input * prime1;
hash = ((hash << 23) | (hash >> (64 - 23))) * prime2 + prime3;
input += 4;
length -= 4;
}
while (length > 0)
{
hash ^= *input * prime5;
hash = ((hash << 11) | (hash >> (64 - 11))) * prime1;
++input;
--length;
}

hash ^= hash >> 33;
hash *= prime2;
hash ^= hash >> 29;
hash *= prime3;
hash ^= hash >> 32;

return hash;
}
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static uint Hash32(this byte[] buffer)
{
int length = buffer.Length;
unsafe
{
fixed (byte* pointer = buffer)
{
return Hash32(pointer, length);
}
}
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static uint Hash32(this string text) => Hash32(Encoding.UTF8.GetBytes(text));

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static uint Hash32(this Type type) => Hash32(type.FullName);

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static uint Hash32<T>() => Hash32(typeof(T));

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static ulong Hash64(this byte[] buffer)
{
int length = buffer.Length;
unsafe
{
fixed (byte* pointer = buffer)
{
return Hash64(pointer, length);
}
}
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static ulong Hash64(this string text) => Hash64(Encoding.UTF8.GetBytes(text));

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static ulong Hash64(this Type type) => Hash64(type.FullName);

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static ulong Hash64<T>() => Hash64(typeof(T));
}
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 0 additions & 8 deletions com.unity.netcode.gameobjects/Runtime/Hashing/XXHash.meta

This file was deleted.

21 changes: 0 additions & 21 deletions com.unity.netcode.gameobjects/Runtime/Hashing/XXHash/LICENSE

This file was deleted.

This file was deleted.

Loading