Skip to content
This repository was archived by the owner on Jan 15, 2024. It is now read-only.

[REFACTOR] Drop Python 2 support and update pylint #838

Merged
merged 11 commits into from
Jul 19, 2019
Merged

[REFACTOR] Drop Python 2 support and update pylint #838

merged 11 commits into from
Jul 19, 2019

Conversation

leezu
Copy link
Contributor

@leezu leezu commented Jul 18, 2019 •
edited
Loading

Description

Drop Python 2 as per #721.

This allows updating pylint of which our current version emits false positive E1111 with #836.

Also fixes #839.

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Changes

  • Drop Python 2 support and update pylint

@leezu leezu requested a review from szha as a code owner July 18, 2019 09:51
@codecov
Copy link

codecov bot commented Jul 18, 2019

Codecov Report

❗ No coverage uploaded for pull request head (droppy2@06219ac). Click here to learn what that means.
The diff coverage is n/a.

@codecov
Copy link

codecov bot commented Jul 18, 2019 •
edited
Loading

Codecov Report

Merging #838 into master will decrease coverage by 0.25%.
The diff coverage is 92.45%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #838      +/-   ##
==========================================
- Coverage   90.43%   90.18%   -0.26%     
==========================================
  Files          66       66              
  Lines        6348     6345       -3     
==========================================
- Hits         5741     5722      -19     
- Misses        607      623      +16
Impacted Files Coverage Δ
src/gluonnlp/embedding/evaluation.py 95.79% <ø> (-0.11%) ⬇️
src/gluonnlp/utils/parameter.py 81.35% <0%> (ø) ⬆️
src/gluonnlp/data/utils.py 70.74% <100%> (-3.41%) ⬇️
src/gluonnlp/vocab/vocab.py 97.29% <100%> (ø) ⬆️
src/gluonnlp/vocab/subwords.py 81.13% <100%> (-5.67%) ⬇️
src/gluonnlp/utils/files.py 45.09% <100%> (ø) ⬆️
src/gluonnlp/vocab/elmo.py 96.66% <100%> (ø) ⬆️
src/gluonnlp/data/sampler.py 96.48% <100%> (ø) ⬆️
src/gluonnlp/data/batchify/embedding.py 97.65% <100%> (ø) ⬆️
src/gluonnlp/data/dataset.py 99.2% <100%> (ø) ⬆️
... and 16 more

@mli
Copy link
Member

mli commented Jul 18, 2019

Found link check problems in job PR-838/1:
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 7) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 7) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 3) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 19) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 21) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 442) broken https://www.aclweb.org/anthology/P02-1040.pdf)[1 - 404 Client Error: Not Found for url: https://www.aclweb.org/anthology/P02-1040.pdf)%5B1
(line 92) broken https://nlp.stanford.edu/pubs/glove.pdf)[2 - 404 Client Error: NOT FOUND for url: https://nlp.stanford.edu/pubs/glove.pdf)%5B2
(line 208) broken https://www.bioinf.jku.at/publications/older/2604.pdf)[3 - 404 Client Error: Not Found for url: https://www.bioinf.jku.at/publications/older/2604.pdf)%5B3

@mli
Copy link
Member

mli commented Jul 18, 2019

Job PR-838/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-838/1/index.html

@mli
Copy link
Member

mli commented Jul 18, 2019

Found link check problems in job PR-838/3:
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 3) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 7) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 7) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 19) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 21) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 442) broken https://www.aclweb.org/anthology/P02-1040.pdf)[1 - 404 Client Error: Not Found for url: https://www.aclweb.org/anthology/P02-1040.pdf)%5B1
(line 92) broken https://nlp.stanford.edu/pubs/glove.pdf)[2 - 404 Client Error: NOT FOUND for url: https://nlp.stanford.edu/pubs/glove.pdf)%5B2
(line 208) broken https://www.bioinf.jku.at/publications/older/2604.pdf)[3 - 404 Client Error: Not Found for url: https://www.bioinf.jku.at/publications/older/2604.pdf)%5B3

@mli
Copy link
Member

mli commented Jul 18, 2019

Job PR-838/3 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-838/3/index.html

@leezu leezu requested a review from eric-haibin-lin July 18, 2019 14:57
@mli
Copy link
Member

mli commented Jul 18, 2019

Found link check problems in job PR-838/4:
(line 7) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 7) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 3) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 4) broken https://gluebenchmark.com/tasks - 404 Client Error: Not Found for url: https://gluebenchmark.com/tasks
(line 19) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 21) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 442) broken https://www.aclweb.org/anthology/P02-1040.pdf)[1 - 404 Client Error: Not Found for url: https://www.aclweb.org/anthology/P02-1040.pdf)%5B1
(line 92) broken https://nlp.stanford.edu/pubs/glove.pdf)[2 - 404 Client Error: NOT FOUND for url: https://nlp.stanford.edu/pubs/glove.pdf)%5B2
(line 208) broken https://www.bioinf.jku.at/publications/older/2604.pdf)[3 - 404 Client Error: Not Found for url: https://www.bioinf.jku.at/publications/older/2604.pdf)%5B3

@mli
Copy link
Member

mli commented Jul 18, 2019

Job PR-838/4 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-838/4/index.html

@leezu leezu requested a review from sxjscience July 19, 2019 09:29
sxjscience
sxjscience reviewed Jul 19, 2019
View reviewed changes
src/gluonnlp/embedding/evaluation.py
@@ -32,12 +32,10 @@

class _WordEmbeddingEvaluationFunction(HybridBlock): # pylint: disable=abstract-method
"""Base class for word embedding evaluation functions."""
pass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why should we remove the pass here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not required. Lint fails (pylint 2 introduced a new rule)

@leezu leezu merged commit 5dcb86a into dmlc:master Jul 19, 2019
@leezu leezu deleted the droppy2 branch July 19, 2019 20:08
leezu added a commit to leezu/gluon-nlp that referenced this pull request Aug 4, 2019
@leezu
Disable false positive pylint warnings
64b5aa0 
Fixed in master branch by dmlc#838
Fix does not apply here as it would require dropping py2.
leezu added a commit that referenced this pull request Aug 5, 2019
@leezu
Backport #862 (#863)
70c5438 
* Fix Vocab with unknown_token remapped to != 0 via token_to_idx arg

* Add test

* Update test_pretrained_bert_models

Confirmed that vocab[vocab.unknown_token] still == 0 for all models created
prior to the flexible vocab PR. Ie:

- book_corpus_wiki_en_uncased
- wiki_multilingual_uncased
- openwebtext_book_corpus_wiki_en_uncased
- wiki_multilingual_cased
- wiki_cn_cased

* Use cuda 10.1 on CI

* Disable false positive pylint warnings

Fixed in master branch by #838
Fix does not apply here as it would require dropping py2.

* Don't test against v1.6 beta and v1.4.1 at the same time.

Test v0.7x branch only on v1.4 due to incompatible doctest output on both mxnet
versions

* Disable test_finetune_chinese_inference due to DNS error
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@sxjscience sxjscience sxjscience approved these changes

@szha szha szha approved these changes

@eric-haibin-lin eric-haibin-lin Awaiting requested review from eric-haibin-lin

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

MXNet nightly numpy dependency bump breaks stuff
4 participants
@leezu @mli @szha @sxjscience