Gym: Allow visual and vector observations at the same time

[shakenes]

Hey everybody,
I would like to propose to allow visual and vector observations at the same time in the gym interface. This might not make a huge difference for most problems, but sometimes you want to solve a problem using visual data (navigation for example) and you need some additional information (ground truth position of the agent or whatever). For me, it just doesn't make sense to allow only vector or visual observations.

The changes in the code remain few. We would only need to update the _single_step function as done here: https://github.com/shakenes/ml-agents/commit/6e6187e2b56ddf47036a8d940692a5d8c99cc674

Please let me know what you think about this idea.

[awjuliani]

Hi @shakenes

We currently do not support this because we want the gym wrapper to be as compatible with the majority of RL algorithms written for ATARI/Mujoco style environments. Of course, users are free to fork our implementation and make their own, but we wanted what we officially shared to maintain only the basic functionality for optimal compatibility.

[shakenes]

Hey @awjuliani,

thank you for your reply. But to be honest, I don't really understand your argument. Why does the Gym interface need to discard observations? If the RL algorithm that someone wants to use, only supports a single type of observation (which probably most of them do, I admit), one could just implement the environment to only provide this exact type of observations. This is super easy to do in Unity thanks to ml-agents.

But the other way round, if somebody (like me) needs more than one type of observations, they spend quite some time to make everything work. First, you do a lot of research to make sure that this indeed is not possible (it's quite unexpected behaviour for me). Then they try to figure out another way to "hack" this and from now on maintain their own fork.

As far as I understand, adding this feature (or rather removing this restriction) would not influence compatibility to other RL algortithms at all. It would be nice if you can explain this in more detail.

Sidenote: My proposed commit above would indeed break compatibility. But that can be easily resolved.

[awjuliani]

I think the argument you are making is a reasonable one. So long as we can properly convey compatibility to users of the gym wrapper, it makes sense to allow this. If you would like to make a PR, we can review it.

[shakenes]

I opened up a PR, so I'm closing this issue.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.