Skip to content

[CI] Move CPU-only jobs to xenial #24506

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 33 commits into from

Conversation

jamesr66a
Copy link
Collaborator

@jamesr66a jamesr66a commented Aug 16, 2019

Stack from ghstack:

Differential Revision: D16862005

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into -3 instead of -5)

Built on top of #24795 (thanks @kostmo!)

@pytorchbot pytorchbot added the module: ci Related to continuous integration label Aug 16, 2019
@jamesr66a jamesr66a requested a review from ezyang August 16, 2019 18:43
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

This will give us diff signal for CPU-only builds with FBGEMM enabled, ensuring quantization gets timely signal
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

This will give us diff signal for CPU-only builds with FBGEMM enabled, ensuring quantization gets timely signal
jamesr66a pushed a commit that referenced this pull request Aug 16, 2019
ghstack-source-id: cfbccf0
Pull Request resolved: #24506
ezyang
ezyang previously approved these changes Aug 16, 2019
@jamesr66a
Copy link
Collaborator Author

@ezyang FYI this still doesn't build with FBGEMM ostensibly due to a binutils version mismatch in 14.04. I think i'm just going to enable a Xenial CPU configuration instead

Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

This will give us diff signal for CPU-only builds with FBGEMM enabled, ensuring quantization gets timely signal
@jamesr66a jamesr66a changed the title [CI] Enable trusty with GCC 7 on diffs [CI] Enable CPU-only xenial with GCC 7 on diffs Aug 16, 2019
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

This will give us diff signal for CPU-only builds with FBGEMM enabled, ensuring quantization gets timely signal
@jamesr66a jamesr66a dismissed ezyang’s stale review August 16, 2019 20:50

changed a bunch of stuff

Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

This will give us diff signal for CPU-only builds with FBGEMM enabled, ensuring quantization gets timely signal
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

This will give us diff signal for CPU-only builds with FBGEMM enabled, ensuring quantization gets timely signal
@jamesr66a jamesr66a changed the title [CI] Enable CPU-only xenial with GCC 7 on diffs [CI] Move CPU-only jobs to xenial Aug 16, 2019
@jamesr66a jamesr66a requested a review from kostmo August 16, 2019 22:00
@jamesr66a jamesr66a changed the title [CI] Move CPU-only jobs to xenial [WIP][CI] Move CPU-only jobs to xenial Aug 16, 2019
@jamesr66a
Copy link
Collaborator Author

Currently getting this error in the regen script:

+ ./generate_config_yml.py
Traceback (most recent call last):
  File "./generate_config_yml.py", line 127, in <module>
    stitch_sources(sys.stdout)
  File "./generate_config_yml.py", line 122, in stitch_sources
    f.write(output_filehandle)
  File "./generate_config_yml.py", line 43, in write
    self.function(build_dict)
  File "/home/jamesreed/pytorch/.circleci/cimodel/data/pytorch_build_definitions.py", line 272, in add_build_env_defs
    d = c.gen_yaml_tree(phase)
  File "/home/jamesreed/pytorch/.circleci/cimodel/data/pytorch_build_definitions.py", line 80, in gen_yaml_tree
    ("DOCKER_IMAGE", self.gen_docker_image_path()),
  File "/home/jamesreed/pytorch/.circleci/cimodel/data/pytorch_build_definitions.py", line 59, in gen_docker_image_path
    base_build_env_name = "-".join(parms_source.get_parms(True))```

Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

This will give us diff signal for CPU-only builds with FBGEMM enabled, ensuring quantization gets timely signal
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

This will give us diff signal for CPU-only builds with FBGEMM enabled, ensuring quantization gets timely signal

Built on top of #24795 (thanks @kostmo!)
jamesr66a pushed a commit that referenced this pull request Aug 16, 2019
ghstack-source-id: 1daba2f
Pull Request resolved: #24506
@ezyang
Copy link
Contributor

ezyang commented Aug 17, 2019

I updated the ECRs for the renaming to remove gcc7.3 to gcc7

@ezyang
Copy link
Contributor

ezyang commented Aug 17, 2019

@kostmo on the regen script

jamesr66a pushed a commit that referenced this pull request Aug 20, 2019
ghstack-source-id: 1ff4fd8
Pull Request resolved: #24506
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into <current year>-3 instead of <current year>-5)

Built on top of #24795 (thanks @kostmo!)
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into <current year>-3 instead of <current year>-5)

Built on top of #24795 (thanks @kostmo!)
jamesr66a pushed a commit that referenced this pull request Aug 20, 2019
ghstack-source-id: 3a13257
Pull Request resolved: #24506
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into <current year>-3 instead of <current year>-5)

Built on top of #24795 (thanks @kostmo!)
jamesr66a pushed a commit that referenced this pull request Aug 20, 2019
ghstack-source-id: 78d7211
Pull Request resolved: #24506
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into <current year>-3 instead of <current year>-5)

Built on top of #24795 (thanks @kostmo!)
jamesr66a pushed a commit that referenced this pull request Aug 20, 2019
ghstack-source-id: c308929
Pull Request resolved: #24506
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into <current year>-3 instead of <current year>-5)

Built on top of #24795 (thanks @kostmo!)
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into <current year>-3 instead of <current year>-5)

Built on top of #24795 (thanks @kostmo!)
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into <current year>-3 instead of <current year>-5)

Built on top of #24795 (thanks @kostmo!)
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into <current year>-3 instead of <current year>-5)

Built on top of #24795 (thanks @kostmo!)
Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into <current year>-3 instead of <current year>-5)

Built on top of #24795 (thanks @kostmo!)
jamesr66a pushed a commit that referenced this pull request Aug 21, 2019
ghstack-source-id: de71a3f
Pull Request resolved: #24506
@jamesr66a
Copy link
Collaborator Author

@pytorchbot retest this please

Differential Revision: [D16862005](https://our.internmc.facebook.com/intern/diff/D16862005)

Trusty is out of standard support, and a mismatch in the binutils version causes FBGEMM to not build in pytorch even with a compiler that supports AVX512. This makes it so we build all that stuff in the CPU CI jobs (and otherwise brings us into <current year>-3 instead of <current year>-5)

Built on top of #24795 (thanks @kostmo!)
@zou3519 zou3519 deleted the gh/jamesr66a/65/head branch August 22, 2019 01:13
@facebook-github-bot
Copy link
Contributor

@jamesr66a merged this pull request in 4966268.

@ezyang
Copy link
Contributor

ezyang commented Sep 19, 2019

This Docker version upgrade did NOT update https://github.com/pytorch/ossci-job-dsl/blob/master/src/main/groovy/ossci/pytorch/DockerVersion.groovy . When you fail to update this file, the new stable Docker image will get deleted after it becomes two weeks old. Please update this file!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Merged module: ci Related to continuous integration module: internals Related to internal abstractions in c10 and ATen
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants