Skip to content

Fix visual observation tensor indexing for Unity inference #6239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Fix visual observation tensor indexing for Unity inference #6239

wants to merge 1 commit into from

Conversation

pglagol
Copy link

@pglagol pglagol commented Aug 18, 2025

Summary

  • Fix tensor indexing in Unity inference to use correct CHW layout
  • Fixed erroneous behavior of the agent on the Unity side when there is more than one visual channel

Problem

The indexing formula was calculating HWC layout while Unity's visual sensors write data in CHW format.

Solution

Corrected TensorExtensions.Index() to properly calculate CHW tensor indices that match Unity's observation writer
format.

@pglagol
Fix visual observation tensor indexing for Unity inference.
0155f62 
This change corrects the tensor indexing calculation in TensorExtensions.Index()
to properly support CHW (channels-height-width) format used by both Unity's
observation writers and ONNX models during inference.
@pglagol pglagol mentioned this pull request Aug 18, 2025
Open
robinDLFM
robinDLFM approved these changes Aug 21, 2025
View reviewed changes
Copy link

@robinDLFM robinDLFM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solved it for me

@maryamziaa maryamziaa self-requested a review August 22, 2025 17:11
maryamziaa
maryamziaa reviewed Aug 22, 2025
View reviewed changes
Copy link
Contributor

@maryamziaa maryamziaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thank you for submitting the PR. I'll take a look after Release 4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@maryamziaa maryamziaa maryamziaa left review comments

+1 more reviewer

@robinDLFM robinDLFM robinDLFM approved these changes

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pglagol @maryamziaa @robinDLFM