Add types and improve descriptions to ArgumentParser parameters #4724

puhuk · 2021-10-22T12:22:47Z

To resolve issue #4694, add the missing ArgumentParser types

datumbox · 2021-10-22T12:24:42Z

@puhuk Note that there are multiple missing types. I just provide a handful of examples on the issue but it would be good to check all of them. Moreover, it might be good to update the descriptions to some parameters that don't have enough info. Have a look and let me know.

puhuk · 2021-10-22T12:31:47Z

@datumbox, Ah, ok got it :) Let me resend after looking them all.

Add missing types on argument parser

puhuk · 2021-10-22T16:18:52Z

I resend PR but some checks failed. (build, ci/circle: python_lint, nightly)
Is there anything I should do before sending PR?

datumbox · 2021-10-22T16:27:28Z

@puhuk You should run ufmt format references from the root of your repo to format the folder.

@NicolasHug I wonder if there is a way we can make the ufmt test show were the failures occurred. Is this possible?

formatted with ufmt

datumbox

@puhuk Thanks for the PR. I've added a few indicative comments.

I was hoping that we could look holistically the parameters and the descriptions across all scripts (including similarity and video_classification) and ensure that a) the parameters are properly validated by correct types, b) the descriptions are consistent across scripts c) they are intuitive and short. Naming is NP hard, so definitely not an easy task...

Let me know your thoughts.

datumbox · 2021-10-22T16:39:17Z

references/classification/train.py

-    parser.add_argument("--device", default="cuda", help="device")
+    parser.add_argument("--data-path", default="/datasets01/imagenet_full_size/061417/", type=str, help="dataset path")
+    parser.add_argument("--model", default="resnet18", type=str, help="model name")
+    parser.add_argument("--device", default="cuda", type=str, help="device (cuda for GPU use, cpu for not using GPU")


Perhaps the following is clearer:

Suggested change

parser.add_argument("--device", default="cuda", type=str, help="device (cuda for GPU use, cpu for not using GPU")

parser.add_argument("--device", default="cuda", type=str, help="device (Use cuda or cpu Default: cuda")

datumbox · 2021-10-22T16:40:50Z

references/classification/train.py

-    parser.add_argument("--output-dir", default=".", help="path where to save")
-    parser.add_argument("--resume", default="", help="resume from checkpoint")
+    parser.add_argument("--output-dir", default=".", type=str, help="path to save outputs")
+    parser.add_argument("--resume", default="", type=str, help="path for resume checkpoint from")


Perhaps:

Suggested change

parser.add_argument("--resume", default="", type=str, help="path for resume checkpoint from")

parser.add_argument("--resume", default="", type=str, help="path of checkpoint")

datumbox · 2021-10-22T16:41:18Z

references/detection/train.py

+    parser.add_argument("--data-path", default="/datasets01/COCO/022719/", type=str, help="dataset path")
+    parser.add_argument("--dataset", default="coco", type=str, help="dataset name")
+    parser.add_argument("--model", default="maskrcnn_resnet50_fpn", type=str, help="model name")
+    parser.add_argument("--device", default="cuda", type=str, help="device (cuda for GPU use, cpu for not using GPU")


same as above

datumbox · 2021-10-22T16:41:33Z

references/detection/train.py

-    parser.add_argument("--output-dir", default=".", help="path where to save")
-    parser.add_argument("--resume", default="", help="resume from checkpoint")
+    parser.add_argument("--output-dir", default=".", type=str, help="path where to save")
+    parser.add_argument("--resume", default="", type=str, help="path for resume checkpoint from")


same as above

datumbox · 2021-10-22T16:42:09Z

references/detection/train.py

@@ -92,15 +94,17 @@ def get_args_parser(add_help=True):
        "--lr-gamma", default=0.1, type=float, help="decrease lr by a factor of lr-gamma (multisteplr scheduler only)"
    )
    parser.add_argument("--print-freq", default=20, type=int, help="print frequency")
-    parser.add_argument("--output-dir", default=".", help="path where to save")
-    parser.add_argument("--resume", default="", help="resume from checkpoint")
+    parser.add_argument("--output-dir", default=".", type=str, help="path where to save")


I think you missed your earlier proposal for different help message here.

datumbox · 2021-10-22T16:42:38Z

references/segmentation/train.py

    parser.add_argument("--aux-loss", action="store_true", help="auxiliar loss")
-    parser.add_argument("--device", default="cuda", help="device")
-    parser.add_argument("-b", "--batch-size", default=8, type=int)
+    parser.add_argument("--device", default="cuda", type=str, help="device (cuda for GPU use, cpu for not using GPU")


datumbox · 2021-10-22T16:43:45Z

references/segmentation/train.py

-    parser.add_argument("--device", default="cuda", help="device")
-    parser.add_argument("-b", "--batch-size", default=8, type=int)
+    parser.add_argument("--device", default="cuda", type=str, help="device (cuda for GPU use, cpu for not using GPU")
+    parser.add_argument("-b", "--batch-size", default=8, type=int, help="size of batch")


Shall we align this with the values of the other scripts? See detection for instance.

datumbox · 2021-10-22T16:43:55Z

references/segmentation/train.py

-    parser.add_argument("--output-dir", default=".", help="path where to save")
-    parser.add_argument("--resume", default="", help="resume from checkpoint")
+    parser.add_argument("--output-dir", default=".", type=str, help="path where to save")
+    parser.add_argument("--resume", default="", type=str, help="path for resume checkpoint from")


same as above

datumbox · 2021-10-22T16:44:05Z

references/segmentation/train.py

@@ -227,8 +227,8 @@ def get_args_parser(add_help=True):
    parser.add_argument("--lr-warmup-method", default="linear", type=str, help="the warmup method (default: linear)")
    parser.add_argument("--lr-warmup-decay", default=0.01, type=float, help="the decay for lr")
    parser.add_argument("--print-freq", default=10, type=int, help="print frequency")
-    parser.add_argument("--output-dir", default=".", help="path where to save")
-    parser.add_argument("--resume", default="", help="resume from checkpoint")
+    parser.add_argument("--output-dir", default=".", type=str, help="path where to save")


Same as above

Updated with review

puhuk · 2021-10-23T10:50:25Z

I see :)
I send again with a) other files including arguments and b) whether descriptions are consistent between files.
If there is anything I missed, please just let me know.
Looking forward to getting review :)

Add train.py from video_classification, similarity and train_quantization.py

datumbox

LGTM, thanks!

…ers (#4724) Summary: * Add type to default argument To resolve issue #4694 * Resolve issue #4694 Add missing types on argument parser * Update with ufmt formatted with ufmt * Updated with review Updated with review * Update type of arguments Add train.py from video_classification, similarity and train_quantization.py Reviewed By: NicolasHug Differential Revision: D31916335 fbshipit-source-id: a717a3cac868b567db57b84a545ad9363820179b Co-authored-by: Vasilis Vryniotis <[email protected]>

…rch#4724) * Add type to default argument To resolve issue pytorch#4694 * Resolve issue pytorch#4694 Add missing types on argument parser * Update with ufmt formatted with ufmt * Updated with review Updated with review * Update type of arguments Add train.py from video_classification, similarity and train_quantization.py Co-authored-by: Vasilis Vryniotis <[email protected]>

Add type to default argument

2765c99

To resolve issue pytorch#4694

pytorch-probot bot added the ciflow/default label Oct 22, 2021

facebook-github-bot added the cla signed label Oct 22, 2021

Resolve issue pytorch#4694

345f528

Add missing types on argument parser

Update with ufmt

8f73e4f

formatted with ufmt

datumbox reviewed Oct 22, 2021

View reviewed changes

Updated with review

0b50f30

Updated with review

Update type of arguments

cea89eb

Add train.py from video_classification, similarity and train_quantization.py

datumbox changed the title ~~Add type to default argument (To resolve issue #4694)~~ Add types and improve descriptions to ArgumentParser parameters Oct 24, 2021

datumbox added enhancement module: reference scripts labels Oct 24, 2021

datumbox approved these changes Oct 24, 2021

View reviewed changes

Merge branch 'main' into issue_4694

5ed897e

datumbox merged commit 9ae833a into pytorch:main Oct 24, 2021

puhuk deleted the issue_4694 branch November 3, 2021 23:33

	parser.add_argument("--device", default="cuda", type=str, help="device (cuda for GPU use, cpu for not using GPU")
	parser.add_argument("--device", default="cuda", type=str, help="device (Use cuda or cpu Default: cuda")

	parser.add_argument("--resume", default="", type=str, help="path for resume checkpoint from")
	parser.add_argument("--resume", default="", type=str, help="path of checkpoint")

Add types and improve descriptions to ArgumentParser parameters #4724

Add types and improve descriptions to ArgumentParser parameters #4724

Uh oh!

Conversation

puhuk commented Oct 22, 2021 • edited by pytorch-probot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datumbox commented Oct 22, 2021

Uh oh!

puhuk commented Oct 22, 2021

Uh oh!

puhuk commented Oct 22, 2021

Uh oh!

datumbox commented Oct 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datumbox left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

puhuk commented Oct 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datumbox left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

puhuk commented Oct 22, 2021 •

edited by pytorch-probot bot

Loading

datumbox commented Oct 22, 2021 •

edited

Loading

puhuk commented Oct 23, 2021 •

edited

Loading