Fix a bug in CUDA pool op #19780

snnn · 2024-03-05T17:29:11Z

Description

PR #17200 introduced a bug in CUDA EP. Before the change, a piece of code in poo.cc was like:

    // cudnn only takes 4D or 5D input, so pad dimensions if needed
    x_dims_cudnn.push_back(1);
    y_dims_cudnn.push_back(1);
    pads.insert(pads.begin() + kernel_shape.size(), 0);
    pads.insert(pads.end(), 0);
    kernel_shape.push_back(1);
    strides.push_back(1);

After the change, the code becomes:

    if (NHWC) {
      x_dims_cudnn.insert(x_dims_cudnn.begin() + 1, 1);
      y_dims_cudnn.insert(y_dims_cudnn.begin() + 1, 1);
      kernel_shape.insert(kernel_shape.begin() + 1, 1);
      strides.insert(strides.begin() + 1, 1);
    } else {
      x_dims_cudnn.push_back(1);
      y_dims_cudnn.push_back(1);
      kernel_shape.push_back(1);
      strides.push_back(1);
    }
    pads.insert(pads.begin() + kernel_shape.size(), 0);
    pads.insert(pads.end(), 0);

The change swapped the order of "pads.insert(pads.begin() + kernel_shape.size(), 0);" and "kernel_shape.push_back(1);"
Therefore the "pads.insert" could cause an out-of-bounds write.

Motivation and Context

snnn · 2024-03-05T17:29:38Z

onnxruntime/core/providers/cuda/nn/pool.cc

@@ -182,16 +182,20 @@ Status Pool<T, PoolType, NHWC>::ComputeInternal(OpKernelContext* context) const
    if (NHWC) {
      x_dims_cudnn.insert(x_dims_cudnn.begin() + 1, 1);
      y_dims_cudnn.insert(y_dims_cudnn.begin() + 1, 1);
+      ORT_ENFORCE(pads.size() >= kernel_shape.size());


TODO: I'm not sure what's the correct order for this NHWC path.

The NHWC implementation seems not right: x_dims_cudnn.insert(x_dims_cudnn.begin() + 1, 1) means it adds a dimension H (like NWC => NHWC). Then the kernel_shape and pads for spatial axis shall pad to the first instead of second.

I think a proper implementation (NHC => NHWC):

if (NHWC) { x_dims_cudnn.insert(x_dims_cudnn.begin() + 2, 1); y_dims_cudnn.insert(y_dims_cudnn.begin() + 2, 1); ORT_ENFORCE(pads.size() >= kernel_shape.size()); pads.insert(pads.begin() + kernel_shape.size(), 0); pads.insert(pads.end(), 0); kernel_shape.push_back(1); strides.push_back(1); }

Then we can refactor it to share last 5 lines of common code for NHWC and NCHW.

BTW, the code for global pooling also need add NHWC support (in a separated PR):

onnxruntime/onnxruntime/core/providers/cuda/nn/pool.cc

Lines 163 to 168 in 901356d

if (pool_attrs_.global_pooling) {

kernel_shape.assign(x_dims.begin() + 2, x_dims.end());

pads.assign(kernel_shape.size(), 0);

strides.assign(kernel_shape.size(), 1);

}

auto out_channel = NHWC ? x_shape[3] : x_shape[1];

I think pads shall be like pads.assign(kernel_shape.size() * 2, 0); in line 165?
Line 168 does not handle 3D or 5D inputs, could cause issue as well.

Agree. We have been adding the extra feature dim to the right of the existing feature dim - not the to the left.

(1) It seems like - we need 1-D NHWC Pool tests added (would have caught the issue Tianlei mentioned)
(2) The existing 1-D NCHW Pool tests need some enhancements - (

onnxruntime/onnxruntime/test/providers/cpu/nn/pool_op_test.cc

Line 202 in 06e684c

static void MaxPool1D_8_WithIndexTest(int64_t storage_order) {

) seem to only have tests with pads as 0s meaning it can't catch this bug. If we had a 1-D test with pads like {0,1} for example - it would have made pads {0,1,0,0} with the change in the PR instead of the correct {0,0,1,0} with the previous logic.

Will update the code shortly.

BTW, the code for global pooling also need add NHWC support (in a separated PR):

Right. But how could we already have a unit test for it, and it is passing? See this commit: 8e1b4b8
The commit was trying to disable the test.

pranavsharma · 2024-03-05T17:42:25Z

Is there a test case?

snnn · 2024-03-05T18:22:19Z

Is there a test case?

It was an out-of-bounds write.error, which results undefined behavior, which mean a test case often may still pass.

hariharans29 · 2024-03-05T18:58:36Z

CC: @gedoensmax as FYI

prathikr · 2024-03-07T22:51:48Z

FYI: AzureML Vision team is also encountering this issue. Thank you @snnn @hariharans29 and @pranavsharma for investigating.

Repro:

import torch.nn.functional as F
from torch import nn
from onnxruntime.training.ortmodule import ORTModule, DebugOptions, LogLevel
import torch

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(28, 10)
        self.fc2 = nn.Linear(10, 10)
        self.pooler = nn.AdaptiveAvgPool1d(1) # BUG

    def forward(self, x):
        x = self.pooler(x) # BUG
        x = x.view(x.shape[0], -1)
        
        x = F.relu(self.fc1(x))
        x = self.fc2(x)

        return x

model = Net()
model = ORTModule(model, DebugOptions(save_onnx=True, onnx_prefix='pooler', log_level=LogLevel.VERBOSE))
model.to("cuda")

images = torch.randn(8, 28, 28).to("cuda")
output = model(images)

onnxruntime/core/providers/cuda/nn/pool.cc

snnn · 2024-03-11T19:41:27Z

Add a test case. Without the code change in pool.cc, the code fails with

[chasun@bigdisk Debug]$ ./onnxruntime_test_all --gtest_filter=PoolTest.MaxPool1D_case3
Note: Google Test filter = PoolTest.MaxPool1D_case3
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from PoolTest
[ RUN ] PoolTest.MaxPool1D_case3
/data/bt/src/onnxruntime/onnxruntime/test/providers/checkers.cc:300: Failure
The difference between cur_expected[i] and cur_actual[i] is 3.4028234663852886e+38, which exceeds tolerance, where
cur_expected[i] evaluates to 2,
cur_actual[i] evaluates to -3.4028234663852886e+38, and
tolerance evaluates to 0.004999999888241291.
i:0
Google Test trace:
/data/bt/src/onnxruntime/onnxruntime/test/providers/checkers.cc:489: provider type: CUDAExecutionProvider
/data/bt/src/onnxruntime/onnxruntime/test/providers/base_tester.cc:809: registered execution providers: CUDAExecutionProvider
Stack trace:
0xf809b6: onnxruntime::test::(anonymous namespace)::InternalNumericalCheck<>()
0xf7e5fa: onnxruntime::test::(anonymous namespace)::TensorCheck<>::operator()()
0xf82aa5: onnxruntime::utils::mltype_dispatcher_internal::CallableDispatchableHelper::Invoke<>()
0xf81b61: onnxruntime::utils::MLTypeCallDispatcher<>::InvokeWithLeadingTemplateArgs<>()
0xf80f30: onnxruntime::utils::MLTypeCallDispatcher<>::Invoke<>()
0xf7f760: onnxruntime::test::Check<>()
0xf81014: onnxruntime::test::CheckDispatch<>()
0xf7fcb3: onnxruntime::test::CheckOrtValuesAreEqual()
0xf78015: onnxruntime::test::BaseTester::ExecuteModel<>()
0xf7434b: onnxruntime::test::BaseTester::ExecuteModelForEps()
0xf72c85: onnxruntime::test::BaseTester::RunWithConfig()
0xf71806: onnxruntime::test::BaseTester::Run()
0xf716b1: onnxruntime::test::BaseTester::Run()
0x128913e: onnxruntime::test::PoolTest_MaxPool1D_case3_Test::TestBody()
0x2993ae1: testing::internal::HandleSehExceptionsInMethodIfSupported<>()
0x298d64e: testing::internal::HandleExceptionsInMethodIfSupported<>()
0x2973122: testing::Test::Run()
0x2973b10: testing::TestInfo::Run()
... Google Test internal frames ...

/data/bt/src/onnxruntime/onnxruntime/test/providers/checkers.cc:300: Failure
The difference between cur_expected[i] and cur_actual[i] is 3.4028234663852886e+38, which exceeds tolerance, where
cur_expected[i] evaluates to 3,
cur_actual[i] evaluates to -3.4028234663852886e+38, and
tolerance evaluates to 0.004999999888241291.
i:1
Google Test trace:
/data/bt/src/onnxruntime/onnxruntime/test/providers/checkers.cc:489: provider type: CUDAExecutionProvider
/data/bt/src/onnxruntime/onnxruntime/test/providers/base_tester.cc:809: registered execution providers: CUDAExecutionProvider
Stack trace:
0xf809b6: onnxruntime::test::(anonymous namespace)::InternalNumericalCheck<>()
0xf7e5fa: onnxruntime::test::(anonymous namespace)::TensorCheck<>::operator()()
0xf82aa5: onnxruntime::utils::mltype_dispatcher_internal::CallableDispatchableHelper::Invoke<>()
0xf81b61: onnxruntime::utils::MLTypeCallDispatcher<>::InvokeWithLeadingTemplateArgs<>()
0xf80f30: onnxruntime::utils::MLTypeCallDispatcher<>::Invoke<>()
0xf7f760: onnxruntime::test::Check<>()
0xf81014: onnxruntime::test::CheckDispatch<>()
0xf7fcb3: onnxruntime::test::CheckOrtValuesAreEqual()
0xf78015: onnxruntime::test::BaseTester::ExecuteModel<>()
0xf7434b: onnxruntime::test::BaseTester::ExecuteModelForEps()
0xf72c85: onnxruntime::test::BaseTester::RunWithConfig()
0xf71806: onnxruntime::test::BaseTester::Run()
0xf716b1: onnxruntime::test::BaseTester::Run()
0x128913e: onnxruntime::test::PoolTest_MaxPool1D_case3_Test::TestBody()
0x2993ae1: testing::internal::HandleSehExceptionsInMethodIfSupported<>()
0x298d64e: testing::internal::HandleExceptionsInMethodIfSupported<>()
0x2973122: testing::Test::Run()
0x2973b10: testing::TestInfo::Run()
... Google Test internal frames ...

/data/bt/src/onnxruntime/onnxruntime/test/providers/checkers.cc:300: Failure
The difference between cur_expected[i] and cur_actual[i] is 3.4028234663852886e+38, which exceeds tolerance, where
cur_expected[i] evaluates to 4,
cur_actual[i] evaluates to -3.4028234663852886e+38, and
tolerance evaluates to 0.004999999888241291.
i:2
Google Test trace:
/data/bt/src/onnxruntime/onnxruntime/test/providers/checkers.cc:489: provider type: CUDAExecutionProvider
/data/bt/src/onnxruntime/onnxruntime/test/providers/base_tester.cc:809: registered execution providers: CUDAExecutionProvider
Stack trace:
0xf809b6: onnxruntime::test::(anonymous namespace)::InternalNumericalCheck<>()
0xf7e5fa: onnxruntime::test::(anonymous namespace)::TensorCheck<>::operator()()
0xf82aa5: onnxruntime::utils::mltype_dispatcher_internal::CallableDispatchableHelper::Invoke<>()
0xf81b61: onnxruntime::utils::MLTypeCallDispatcher<>::InvokeWithLeadingTemplateArgs<>()
0xf80f30: onnxruntime::utils::MLTypeCallDispatcher<>::Invoke<>()
0xf7f760: onnxruntime::test::Check<>()
0xf81014: onnxruntime::test::CheckDispatch<>()
0xf7fcb3: onnxruntime::test::CheckOrtValuesAreEqual()
0xf78015: onnxruntime::test::BaseTester::ExecuteModel<>()
0xf7434b: onnxruntime::test::BaseTester::ExecuteModelForEps()
0xf72c85: onnxruntime::test::BaseTester::RunWithConfig()
0xf71806: onnxruntime::test::BaseTester::Run()
0xf716b1: onnxruntime::test::BaseTester::Run()
0x128913e: onnxruntime::test::PoolTest_MaxPool1D_case3_Test::TestBody()
0x2993ae1: testing::internal::HandleSehExceptionsInMethodIfSupported<>()
0x298d64e: testing::internal::HandleExceptionsInMethodIfSupported<>()
0x2973122: testing::Test::Run()
0x2973b10: testing::TestInfo::Run()
... Google Test internal frames ...

/data/bt/src/onnxruntime/onnxruntime/test/providers/checkers.cc:300: Failure
The difference between cur_expected[i] and cur_actual[i] is 3.4028234663852886e+38, which exceeds tolerance, where
cur_expected[i] evaluates to 5,
cur_actual[i] evaluates to -3.4028234663852886e+38, and
tolerance evaluates to 0.004999999888241291.
i:3
Google Test trace:
/data/bt/src/onnxruntime/onnxruntime/test/providers/checkers.cc:489: provider type: CUDAExecutionProvider
/data/bt/src/onnxruntime/onnxruntime/test/providers/base_tester.cc:809: registered execution providers: CUDAExecutionProvider
Stack trace:
0xf809b6: onnxruntime::test::(anonymous namespace)::InternalNumericalCheck<>()
0xf7e5fa: onnxruntime::test::(anonymous namespace)::TensorCheck<>::operator()()
0xf82aa5: onnxruntime::utils::mltype_dispatcher_internal::CallableDispatchableHelper::Invoke<>()
0xf81b61: onnxruntime::utils::MLTypeCallDispatcher<>::InvokeWithLeadingTemplateArgs<>()
0xf80f30: onnxruntime::utils::MLTypeCallDispatcher<>::Invoke<>()
0xf7f760: onnxruntime::test::Check<>()
0xf81014: onnxruntime::test::CheckDispatch<>()
0xf7fcb3: onnxruntime::test::CheckOrtValuesAreEqual()
0xf78015: onnxruntime::test::BaseTester::ExecuteModel<>()
0xf7434b: onnxruntime::test::BaseTester::ExecuteModelForEps()
0xf72c85: onnxruntime::test::BaseTester::RunWithConfig()
0xf71806: onnxruntime::test::BaseTester::Run()
0xf716b1: onnxruntime::test::BaseTester::Run()
0x128913e: onnxruntime::test::PoolTest_MaxPool1D_case3_Test::TestBody()
0x2993ae1: testing::internal::HandleSehExceptionsInMethodIfSupported<>()
0x298d64e: testing::internal::HandleExceptionsInMethodIfSupported<>()
0x2973122: testing::Test::Run()
0x2973b10: testing::TestInfo::Run()
... Google Test internal frames ...

/data/bt/src/onnxruntime/onnxruntime/test/providers/checkers.cc:300: Failure
The difference between cur_expected[i] and cur_actual[i] is 3.4028234663852886e+38, which exceeds tolerance, where
cur_expected[i] evaluates to 5,
cur_actual[i] evaluates to -3.4028234663852886e+38, and
tolerance evaluates to 0.004999999888241291.
i:4
Google Test trace:
/data/bt/src/onnxruntime/onnxruntime/test/providers/checkers.cc:489: provider type: CUDAExecutionProvider
/data/bt/src/onnxruntime/onnxruntime/test/providers/base_tester.cc:809: registered execution providers: CUDAExecutionProvider
Stack trace:
0xf809b6: onnxruntime::test::(anonymous namespace)::InternalNumericalCheck<>()
0xf7e5fa: onnxruntime::test::(anonymous namespace)::TensorCheck<>::operator()()
0xf82aa5: onnxruntime::utils::mltype_dispatcher_internal::CallableDispatchableHelper::Invoke<>()
0xf81b61: onnxruntime::utils::MLTypeCallDispatcher<>::InvokeWithLeadingTemplateArgs<>()
0xf80f30: onnxruntime::utils::MLTypeCallDispatcher<>::Invoke<>()
0xf7f760: onnxruntime::test::Check<>()
0xf81014: onnxruntime::test::CheckDispatch<>()
0xf7fcb3: onnxruntime::test::CheckOrtValuesAreEqual()
0xf78015: onnxruntime::test::BaseTester::ExecuteModel<>()
0xf7434b: onnxruntime::test::BaseTester::ExecuteModelForEps()
0xf72c85: onnxruntime::test::BaseTester::RunWithConfig()
0xf71806: onnxruntime::test::BaseTester::Run()
0xf716b1: onnxruntime::test::BaseTester::Run()
0x128913e: onnxruntime::test::PoolTest_MaxPool1D_case3_Test::TestBody()
0x2993ae1: testing::internal::HandleSehExceptionsInMethodIfSupported<>()
0x298d64e: testing::internal::HandleExceptionsInMethodIfSupported<>()
0x2973122: testing::Test::Run()
0x2973b10: testing::TestInfo::Run()
... Google Test internal frames ...

[ FAILED ] PoolTest.MaxPool1D_case3 (147 ms)
[----------] 1 test from PoolTest (147 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (147 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] PoolTest.MaxPool1D_case3

onnxruntime/test/providers/cpu/nn/pool_op_test.cc

onnxruntime/core/providers/cuda/nn/pool.cc

This reverts commit 1d925f1.

snnn · 2024-03-13T18:24:13Z

This PR is ready to be merged.

hariharans29 · 2024-03-13T20:56:31Z

onnxruntime/test/providers/cuda/nhwc/pool_test.cc

@@ -59,6 +59,8 @@ TYPED_TEST(CudaNhwcTypedTest, MaxPoolNhwc) {
  MAKE_PROVIDERS()
 }

+#if 0


Don't we want to keep this test ? It is 2-D NHWC Global Pooling (not 1-D which we disabled support for)

I thought I reverted it. Thanks for pointing it out.

It looks like the NHWC 2D global pool is still not right. I am debugging it.

tianleiwu · 2024-03-14T00:40:08Z

onnxruntime/core/providers/cuda/nn/pool.cc

+      // The last dim of x_dims is channel(C).
+      // Put the other part in kernel_shape
+      kernel_shape.assign(x_dims.begin() + 1, x_dims.end() - 1);
+      pads.assign(kernel_shape.size(), 0);


pads need to be 2X size of kernel_shape see line 203

Then how could the old code work?

if (pool_attrs_.global_pooling) { kernel_shape.assign(x_dims.begin() + 2, x_dims.end()); pads.assign(kernel_shape.size(), 0); strides.assign(kernel_shape.size(), 1); }

We have test case for that? How did it pass?

AFAIK there are no "pads" for Global pooling - https://onnx.ai/onnx/operators/onnx__GlobalMaxPool.html (i.e.) pads are irrelevant

@snnn, that's discussed in #19889 (comment). I think that PR is almost ready since tests have passed (need some minor change to address build and feedback). That PR can address this bug as well.
The only concern is that PR might depend on other NHWC change, so it might be little harder to cherry-pick if we want to have a patch release soon.

Maybe we can just fix the NCHW regression (for the patch) and leave anything related to NHWC un-implemented that can be filled in by the other PR. In any case, I don't think we have a ton of NHWC users just yet that will be broken by the patch release.

I think either is fine. If we gave up this one, we should cherry-pick #19889 to the patch release. That's PR is not very big. So I am not worried.

After the bugs are fixed ,we should refactor the SetPoolingNdDescriptorHelper function a little: change the three array type parameters to std::span, and check if all their lengths equal. I can take the work after #19889 is merged.

And in CUDAExecutionProvider::GetCapability function https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/providers/cuda/cuda_execution_provider.cc we should check if paddings are symmetric when the op is a pooling op but it is not global pooling.

snnn · 2024-03-15T19:37:25Z

Replaced by #19889

Fix a bug in CUDA pool op

901356d

snnn commented Mar 5, 2024

View reviewed changes

snnn linked an issue Mar 5, 2024 that may be closed by this pull request

[CUDA][1.17.1][Regression] Access Violation in an inference session Run #19778

Closed

snnn requested review from pranavsharma and hariharans29 March 5, 2024 17:35

snnn added the ep:CUDA issues related to the CUDA execution provider label Mar 5, 2024

sophies927 added the release:1.17.3 label Mar 7, 2024

snnn added 2 commits March 11, 2024 10:39

Merge remote-tracking branch 'upstream/main' into fix_bug3

a613fc3

update

115b76d

hariharans29 reviewed Mar 11, 2024

View reviewed changes

onnxruntime/core/providers/cuda/nn/pool.cc Outdated Show resolved Hide resolved

snnn added 2 commits March 11, 2024 12:38

update

1836db5

Merge remote-tracking branch 'upstream/main' into fix_bug3

3822bee

tianleiwu reviewed Mar 11, 2024

View reviewed changes

onnxruntime/test/providers/cpu/nn/pool_op_test.cc Show resolved Hide resolved

update

9eaca8f

hariharans29 reviewed Mar 11, 2024

View reviewed changes

onnxruntime/core/providers/cuda/nn/pool.cc Outdated Show resolved Hide resolved

hariharans29 previously approved these changes Mar 11, 2024

View reviewed changes

fix build

1d925f1

snnn dismissed hariharans29’s stale review via 1d925f1 March 11, 2024 23:19

hariharans29 previously approved these changes Mar 11, 2024

View reviewed changes

Update pool_test.cc

8e1b4b8

snnn dismissed hariharans29’s stale review via 8e1b4b8 March 12, 2024 02:21

hariharans29 reviewed Mar 12, 2024

View reviewed changes

onnxruntime/core/providers/cuda/nn/pool.cc Show resolved Hide resolved

snnn added 3 commits March 12, 2024 10:17

Revert "fix build"

594a89e

This reverts commit 1d925f1.

update

654e53b

update

78dc44d

hariharans29 mentioned this pull request Mar 13, 2024

Fix broken Pooling CUDA NHWC Ops and ensure NCHW / NHWC parity. #19889

Merged

snnn requested a review from hariharans29 March 13, 2024 18:23

prathikr previously approved these changes Mar 13, 2024

View reviewed changes

snnn requested a review from tianleiwu March 13, 2024 19:40

hariharans29 reviewed Mar 13, 2024

View reviewed changes

snnn added 2 commits March 13, 2024 14:09

revert

0e85916

Merge remote-tracking branch 'origin/main' into fix_bug3

c1aec15

snnn dismissed prathikr’s stale review via c1aec15 March 13, 2024 21:09

Merge remote-tracking branch 'origin/main' into fix_bug3

387a6a3

hariharans29 previously approved these changes Mar 13, 2024

View reviewed changes

tianleiwu reviewed Mar 14, 2024

View reviewed changes

update

a10f101

snnn dismissed hariharans29’s stale review via a10f101 March 14, 2024 23:47

snnn closed this Mar 15, 2024

snnn removed the release:1.17.3 label Mar 15, 2024

snnn deleted the fix_bug3 branch June 19, 2024 21:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix a bug in CUDA pool op #19780

Fix a bug in CUDA pool op #19780

snnn commented Mar 5, 2024 •

edited

Loading

snnn Mar 5, 2024

tianleiwu Mar 5, 2024 •

edited

Loading

hariharans29 Mar 5, 2024 •

edited

Loading

snnn Mar 7, 2024

snnn Mar 12, 2024

pranavsharma commented Mar 5, 2024

snnn commented Mar 5, 2024

hariharans29 commented Mar 5, 2024

prathikr commented Mar 7, 2024 •

edited

Loading

snnn commented Mar 11, 2024

snnn commented Mar 13, 2024

hariharans29 Mar 13, 2024 •

edited

Loading

snnn Mar 13, 2024

snnn Mar 13, 2024

tianleiwu Mar 14, 2024 •

edited

Loading

snnn Mar 14, 2024

snnn Mar 14, 2024

snnn Mar 14, 2024

hariharans29 Mar 14, 2024 •

edited

Loading

tianleiwu Mar 15, 2024 •

edited

Loading

hariharans29 Mar 15, 2024

snnn Mar 15, 2024

snnn Mar 15, 2024

snnn Mar 15, 2024

snnn commented Mar 15, 2024

	if (pool_attrs_.global_pooling) {
	kernel_shape.assign(x_dims.begin() + 2, x_dims.end());
	pads.assign(kernel_shape.size(), 0);
	strides.assign(kernel_shape.size(), 1);
	}
	auto out_channel = NHWC ? x_shape[3] : x_shape[1];

Fix a bug in CUDA pool op #19780

Fix a bug in CUDA pool op #19780

Conversation

snnn commented Mar 5, 2024 • edited Loading

Description

Motivation and Context

Choose a reason for hiding this comment

tianleiwu Mar 5, 2024 • edited Loading

Choose a reason for hiding this comment

hariharans29 Mar 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pranavsharma commented Mar 5, 2024

snnn commented Mar 5, 2024

hariharans29 commented Mar 5, 2024

prathikr commented Mar 7, 2024 • edited Loading

snnn commented Mar 11, 2024

snnn commented Mar 13, 2024

hariharans29 Mar 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tianleiwu Mar 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hariharans29 Mar 14, 2024 • edited Loading

Choose a reason for hiding this comment

tianleiwu Mar 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

snnn commented Mar 15, 2024

snnn commented Mar 5, 2024 •

edited

Loading

tianleiwu Mar 5, 2024 •

edited

Loading

hariharans29 Mar 5, 2024 •

edited

Loading

prathikr commented Mar 7, 2024 •

edited

Loading

hariharans29 Mar 13, 2024 •

edited

Loading

tianleiwu Mar 14, 2024 •

edited

Loading

hariharans29 Mar 14, 2024 •

edited

Loading

tianleiwu Mar 15, 2024 •

edited

Loading