Add frozen flag to VisualFoodCollector

[dongruoping]

Proposed change(s)

The FoodCollector environment does not train with pure visual observation due to insufficient information. The agent lacks of information about the frozen state of itself, and adding a boolean flag indicating the frozen state (which is also used in the vector observation version of FoodCollector) makes the training work.

This will add an example of using both vector and visual observation to our example environments.

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

• Bug fix
• New feature
• Code refactor
• Breaking change
• Documentation update
• Other (please describe)

Checklist

• Added tests that prove my fix is effective or that my feature works
• Updated the changelog (if applicable)
• Updated the documentation (if applicable)
• Updated the migration guide (if applicable)

Other comments

[dongruoping]

Docs and changelogs WIP

[vincentpierre]

If you verified that it trains, this is great news!
I left a comment, please address before merging.
do you plan on adding this scene to the daily CI?

[vincentpierre]

I think this should always be turned on. The user might have issues with the number of observations when unchecking this box. Besides, you mentioned that the Agent will not train without it, so I think it should be always on and un-uncheckable.

[dongruoping]

My concern was that it would be a bit confusing to use one vector flag by default in a "visual" scene.
Or is it actually not too big a problem? if so I can make it un-uncheckable.

[dongruoping]

Yes I think it would be good to be include in CI, so that we can capture it if we change the way combining vector and visual observations. It could take quite some time though.
FYI: Using the current config, the reward bounce between -1~1 in the first 1.5M steps, get to ~20 at around 2M steps and ~50 around 3M steps.

[dongruoping]

Added the trained model and updated the docs, so I think it'd be better to do re-review

[vincentpierre]

Looks, good to me. I still think the useFrozenFlag should not be an option and should always be on. I tried the model and it worked. I did not try to train it.

[dongruoping]

Looks, good to me. I still think the useFrozenFlag should not be an option and should always be on. I tried the model and it worked. I did not try to train it.

Removed useFrozenFlag as suggested and verified the change does not affect the model.