Unity hangs indefinitely when Academy.Instance.EnvironmentStep() called on turn-based game

[applenicks]

I am trying to create the right environment to train ML agents on a simple two-player Noughts & Crosses/tic-tac-toe game, and am transitioning from the default auto-step environment to the manually-called Academy.Instance.EnvironmentStep(), as being more suitable for turn-based games. When I include the EnvironmentStep line and press play, unity simply goes into a hang state and needs a force quit (even if I wrap that code into a coroutine).

Reproducing:

• behavior type set to Heuristic for both players (pre training), with random moves programmed
• Academy.Instance.EnvironmentStep() is called every time the next player has a turn
• if I apply it to the first player only (ie comment out if player 2) - it correctly runs and plays a move (before stopping on player 2 with no signal to prompt the next move)
• as soon as I apply it to both players, and press play, the game hangs, not even running

Environment:
Unity 2019.3.15f1
MLAgents 0.17.0
Tensorflow 1.5.0
Mac OS 10.12.3

[andrewcoh]

Hi @applenicks

Can you copy the entire function where you call EnvironmentStep?

[applenicks]

Here's the function NextPlayerTurn, which itself is triggered by an Action:

private void NextPlayerTurn(bool isP1Turn) { noMovesAllowed = false; if (isPlayer1 && isP1Turn) { this.RequestDecision(); Academy.Instance.EnvironmentStep(); } else if (!isPlayer1 && !isP1Turn)// this must be player 2 and it must be player 2's turn { this.RequestDecision(); Academy.Instance.EnvironmentStep(); //This will crash/hang unity! } }

[andrewcoh]

Is this function being manually called (either directly or indirectly) by CollectObservations or OnActionReceived? This can cause a loop that will prevent the main Update from being called.

[applenicks]

Solved - thank you.

Yes: the function is triggered by an Action that is ultimately called by OnActionReceived, which must lead to the infinite loop described in your branched conversation.

I've since prevented OnActionReceived calling EnvironmentStep() directly, and instead have it toggle the noMovesAllows boolean that's now checked on the independent FixedUpdate loop, which now calls the EnvironmentStep().

Thank you.

[andrewcoh]

Great! Additionally, we've added a catch for this today in #4227. Thank you for raising the issue. I am closing this issue as this has been resolved.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.