Skip to content

AWS S3 Tests start to failing due to ValueError: Access Denied #460

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ejguan opened this issue May 25, 2022 · 2 comments
Closed

AWS S3 Tests start to failing due to ValueError: Access Denied #460

ejguan opened this issue May 25, 2022 · 2 comments

Comments

@ejguan
Copy link
Contributor

ejguan commented May 25, 2022

🐛 Describe the bug

Build TorchData with AWSSDK enabled: BUILD_S3=1 python setup.py install
Run test: cpurun python test/test_remote_io.py -v -k test_s3_io_iterdatapipe and you will see ValueError: Access Denied

I will disable the test to make sure CI is green.

cc: @ydaiming
Could you please take a deeper look on the test case?

Versions

main branch
@ejguan
Copy link
Contributor Author

ejguan commented May 25, 2022

There are a few places yielding ValueError: No response body

@ejguan ejguan changed the title S3 Tests start to failing due to ValueError: Access Denied AWS S3 Tests start to failing due to ValueError: Access Denied May 25, 2022
@ejguan ejguan mentioned this issue May 25, 2022
Closed
facebook-github-bot pushed a commit that referenced this issue May 25, 2022
@ejguan @facebook-github-bot
Temporarily disable AWSSDK functional tests (#461)
ec83d11 
Summary:
This PR will temporarily disable the tests for AWSSDK due to #460

The actual fix requires AWS team's contribution.

Pull Request resolved: #461

Reviewed By: NivekT

Differential Revision: D36664896

Pulled By: ejguan

fbshipit-source-id: 65862f44349983afe0d358b62f2dd7999b5a4649
ejguan added a commit to ejguan/data that referenced this issue May 25, 2022
@ejguan
Temporarily disable AWSSDK functional tests (pytorch#461)
6ec85a2 
Summary:
This PR will temporarily disable the tests for AWSSDK due to pytorch#460

The actual fix requires AWS team's contribution.

Pull Request resolved: pytorch#461

Reviewed By: NivekT

Differential Revision: D36664896

Pulled By: ejguan

fbshipit-source-id: 65862f44349983afe0d358b62f2dd7999b5a4649
ejguan added a commit that referenced this issue May 25, 2022
@ejguan
Temporarily disable AWSSDK functional tests (#461)
db3a389 
Summary:
This PR will temporarily disable the tests for AWSSDK due to #460

The actual fix requires AWS team's contribution.

Pull Request resolved: #461

Reviewed By: NivekT

Differential Revision: D36664896

Pulled By: ejguan

fbshipit-source-id: 65862f44349983afe0d358b62f2dd7999b5a4649
@ydaiming ydaiming mentioned this issue May 25, 2022
Closed
@ejguan
Copy link
Contributor Author

ejguan commented May 25, 2022

In order to prevent such issue happening again, @ydaiming suggests to host a few specific public buckets for testing.

ejguan pushed a commit to ejguan/data that referenced this issue May 26, 2022
@ydaiming @ejguan
update s3 test cases (pytorch#464)
b37102c 
Summary:
Please read through our [contribution guide](https://github.com/pytorch/data/blob/main/CONTRIBUTING.md) prior to
creating your pull request.

- Note that there is a section on requirements related to adding a new DataPipe.

Fixes pytorch#460

### Changes

- update the s3 test cases due to an update in the public dataset

Pull Request resolved: pytorch#464

Reviewed By: NivekT

Differential Revision: D36678285

Pulled By: ejguan

fbshipit-source-id: 5e579d20290aec0f4f0a17031fe1bb256a640231
ejguan pushed a commit to ejguan/data that referenced this issue May 26, 2022
@ydaiming @ejguan
update s3 test cases (pytorch#464)
dfb88e3 
Summary:
Please read through our [contribution guide](https://github.com/pytorch/data/blob/main/CONTRIBUTING.md) prior to
creating your pull request.

- Note that there is a section on requirements related to adding a new DataPipe.

Fixes pytorch#460

### Changes

- update the s3 test cases due to an update in the public dataset

Pull Request resolved: pytorch#464

Reviewed By: NivekT

Differential Revision: D36678285

Pulled By: ejguan

fbshipit-source-id: 5e579d20290aec0f4f0a17031fe1bb256a640231
@ejguan ejguan mentioned this issue May 26, 2022
Merged
ejguan pushed a commit that referenced this issue May 26, 2022
@ydaiming @ejguan
update s3 test cases (#464)
3d46ef3 
Summary:
Please read through our [contribution guide](https://github.com/pytorch/data/blob/main/CONTRIBUTING.md) prior to
creating your pull request.

- Note that there is a section on requirements related to adding a new DataPipe.

Fixes #460

### Changes

- update the s3 test cases due to an update in the public dataset

Pull Request resolved: #464

Reviewed By: NivekT

Differential Revision: D36678285

Pulled By: ejguan

fbshipit-source-id: 5e579d20290aec0f4f0a17031fe1bb256a640231
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

update s3 test cases ydaiming/data
1 participant
@ejguan