Skip to content

Conversation

NoelStephensUnity
Copy link
Collaborator

@NoelStephensUnity NoelStephensUnity commented May 18, 2021

This work is based off of MTT-764.
It includes the conversion of the manual RpcTests over to an automated unit test with 1 Host and 7 Clients (8 clients total).
Summary
This task is to convert the manual RpcTesting tests over to an automated unit test leveraging from the new MultiInstanceHelpers class. This includes the refactoring of the RpcQueueManualTests component to be able to handle both Manual and Automated testing.

Acceptance Criteria
The unit test should perform at least one full cycle of the various RPC tests performed by the manual test using 1 Host and 7 clients for a total of 1 Server and 8 clients. The RPC tests performed can be verified by the final output of the unit test to assure all runtime RPC related counters reflect that each type of test has been counted for each client and server multiple times (it varies depending upon the test type). The only value that will not be conveyed is the progress for each multi-client striped test. That is verified by the fact that the test completed successfully.

NoelStephensUnity and others added 28 commits May 13, 2021 16:44
This is the first pass version with a duplicated MultiInstanceHelper.
This includes SIPTransport updates that prevent issues with multiple SIPTransport unit tests as the S_Server was never getting reset.  So, upon the host-server disconnecting it resets the value back to null so the next unit test can initialize the SIPTransport again.

This includes a fix for unit tests that may result in the SnapshotSystem attempting to send before m_NetworkManager is set.
fixing the spaces.
Removing the temporary MultiInstanceHelpers class and migrating the changes over to the original MultiInstanceHelpers class.

Renamed the UnityTest to ManualRpcTestsAutomated.
Including the assembly definition file... :)
Increasing the total number of clients to 10.
1 Host and 9 clients.
Updating comment to reflect the proper number of clients (changed from 7 and 1 host to 9 and 1 host)
Adding the ability to run this same test under different conditions.
This pass provides the ability to specify the number of clients (always will be a host) and whether it will be using batched RPCs or not.
Creating a HybridScripts folder to distinguish between manual tests that have yet to be automated and manual tests that can be run in both a unit test and as a manual test.
Reverting change to snapshot system.
Condensed assembly definition per suggestion by Fatih.
Testing if this change will fix the issue with Ubuntu builds...thinking it was timing out and never cleaning the MultiInstanceHelpers.
Method name change
Now uses frame count to detect a timeout as opposed to using time in the event a virtual machine, during Yamato testing, is under a heavy load and frame per second are much lower than expected.
This also includes changes in the RpcQueueManualTests to help reduce the time a full test will take.
Checking to see if Rpc Manual tests run fine with these changes.
Assuring the  m_ContinueToRun is set to true when setting the test mode (could be that the Start method has yet to be invoked by the time it checks.
Adding a time sanity check...
Running some additional information gathering tests regarding frame rates:.
Did some final clean up on the code to prepare for review.
Cleaned up/added comments.

Added an Application.targetFrameRate check that will assure this test, for all platforms, runs at the same frame update rate.
/// <param name="server">The server</param>
/// <param name="result">The result. If null, it will automatically assert</param>
/// <param name="maxFrames">The max frames to wait for</param>
public static IEnumerator WaitForClientsConnectedToServer(NetworkManager server, int clientCount, CoroutineResultWrapper<bool> result = null, int maxFrames = 64)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have a duplicate method for this? Why not just have a default clientCount=1?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I just made some adjustments which will remove the duplicate code while still keeping the methods in-tact.
( Albin was the original author so I will leave it up to him to determine if we should completely remove the "wrapper methods")

/// <param name="result">The result. If null, it will automatically assert<</param>
/// <param name="maxFrames">The max frames to wait for</param>
/// <returns></returns>
public static IEnumerator WaitForClientsConnected(NetworkManager[] clients, CoroutineResultWrapper<bool> result = null, int maxFrames = 64)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a lot of duplicated logic, isn't there a way to reuse the other WaitForClientsConnected methods?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I just made some adjustments which will remove the duplicate code while still keeping the methods in-tact.
( Albin was the original author so I will leave it up to him to determine if we should completely remove the "wrapper methods")

Sam made a good point about two methods that really were duplicating code.
Made some adjustments that will collapse the duplicated code while also keeping the same methods around.
}

/// <summary>
/// ***Should always be invoked when finished with a single unit test***
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// ***Should always be invoked when finished with a single unit test***
/// Should always be invoked when finished with a single unit test

int startFrame = Time.frameCount;

while (Time.frameCount - startFrame <= maxFrames && !client.IsConnectedClient)
var startFrame = Time.frameCount;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
var startFrame = Time.frameCount;
var startFrameNumber = Time.frameCount;

}

int startFrame = Time.frameCount;
var startFrame = Time.frameCount;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
var startFrame = Time.frameCount;
var startFrameNumber = Time.frameCount;

Copy link
Contributor

@0xFA11 0xFA11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with some minor suggestions, it's gtg! 🚀

@NoelStephensUnity NoelStephensUnity merged commit 77013b4 into develop May 20, 2021
@NoelStephensUnity NoelStephensUnity deleted the test/manualRpcTestsToAutomated-Fix branch May 20, 2021 15:15
SamuelBellomo added a commit that referenced this pull request May 21, 2021
…player-object-get

* develop:
  test: manual connection approval to fully automated connection approval testing (#839)
  test: manual rpc tests to automated fixed revision (#841)
  fix: CleanDiffedSceneObjects was not clearing PendingSoftSyncObjects  (#834)
  feat: NetworkTransform now uses NetworkVariables instead of RPCs (#826)
  fix: gracefully handle exceptions in an RPC invoke (#846)
  fix: Fixing utp license file for legal (#843)
  ci: enable standards check on UTP too (#837)
  refactor: NetworkBehaviour.NetworkObject no longer throws an exception (#838)
  fix: revert #830 (#840)
  test: converting the manual rpc tests over to an automated unit test (#830)

# Conflicts:
#	com.unity.multiplayer.mlapi/Tests/Runtime/MultiInstance/MultiInstanceHelpers.cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants