Image Classification Sample cleanup #4380

ashbhandare · 2019-10-24T22:36:41Z

Make samples more readable and add comments.

codecov · 2019-10-24T23:33:45Z

Codecov Report

Merging #4380 into master will decrease coverage by 15.4%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master    #4380       +/-   ##
===========================================
- Coverage   90.11%    74.7%   -15.41%     
===========================================
  Files         200      906      +706     
  Lines       36848   159288   +122440     
  Branches     1910    17145    +15235     
===========================================
+ Hits        33206   119002    +85796     
- Misses       3337    35476    +32139     
- Partials      305     4810     +4505

Flag	Coverage Δ
#Debug	`74.7% <ø> (-15.41%)`	⬇️
#production	`70.07% <ø> (?)`
#test	`90.11% <ø> (ø)`	⬆️

Impacted Files	Coverage Δ
src/Microsoft.ML.Data/Model/Pfa/ModelUtils.cs	`0% <0%> (ø)`
...ML.Data/Transforms/ColumnConcatenatingEstimator.cs	`83.58% <0%> (ø)`
src/Microsoft.ML.Data/Model/Pfa/PfaUtils.cs	`0% <0%> (ø)`
...t.ML.Data/DataLoadSave/CompositeLoaderEstimator.cs	`80% <0%> (ø)`
...t.ML.Data/DataLoadSave/Text/LoadColumnAttribute.cs	`100% <0%> (ø)`
...ft.ML.Data/DataLoadSave/Database/DatabaseLoader.cs	`52.66% <0%> (ø)`
src/Microsoft.ML.Core/Data/ModelSaveContext.cs	`91.57% <0%> (ø)`
...er/Multiclass/EnsembleMulticlassModelParameters.cs	`84.4% <0%> (ø)`
...crosoft.ML.Transforms/EntryPoints/TextAnalytics.cs	`41.25% <0%> (ø)`
...Microsoft.ML.Data/Transforms/NormalizeColumnDbl.cs	`66.3% <0%> (ø)`
... and 696 more

codemzs · 2019-10-25T23:20:26Z

...classClassification/ImageClassification/LearningRateSchedulingCifarResnetTransferLearning.cs

            string imagesDownloadFolderPath = Path.Combine(assetsPath, "inputs",
                "images");

-            // Download Cifar Dataset. 
+            // Download Cifar Dataset and set train and test dataset
+            // paths 


period #Resolved

codemzs · 2019-10-25T23:24:53Z

...ers/MulticlassClassification/ImageClassification/ResnetV2101TransferLearningEarlyStopping.cs

+                //      Cross-Entropy: 0.03978536
+                // Phase: Training, Dataset used: Validation, Batch Processed Count: 3,
+                //      Epoch: 19, Accuracy: 0.852381
+                // We see that the training stops when the metric stops improving.


Thanks, @[email protected] ! In this case where limiting the output to 85 characters actually makes it difficult to read sample output it is ok to cross 85 character. Otherwise your change looks very good, please fix similar stuff else where. #Resolved

codemzs · 2019-10-25T23:25:20Z

...rs/MulticlassClassification/ImageClassification/ResnetV2101TransferLearningTrainTestSplit.cs

+                //    Learning Rate:       0.01 Epoch:  49, Accuracy:          1, 
+                //    Cross-Entropy: 0.01758162
+                // Phase: Training, Dataset used: Validation, Batch Processed Count: 3, 
+                //    Epoch: 49, Accuracy: 0.8666666


please fix #Resolved

codemzs · 2019-10-25T23:25:39Z

...classClassification/ImageClassification/LearningRateSchedulingCifarResnetTransferLearning.cs

+                // Phase: Training, Dataset used:      Train, Batch Processed Count:  18,
+                //    Learning Rate:       0.01 Epoch:   0, Accuracy:  0.9166667, 
+                //    Cross-Entropy:  0.4866541
+                // ...


please fix #Resolved

…ample

ashbhandare · 2019-11-07T01:38:46Z

.../Dynamic/Trainers/MulticlassClassification/ImageClassification/ImageClassificationDefault.cs

+                "pre-trained model/architecture ***");
+
+            // Measuring training time.
+            var watch = System.Diagnostics.Stopwatch.StartNew();


var watch = System.Diagnostics.Stopwatch.StartNew(); [](start = 12, length = 52)

remove measuring time in samples #Resolved

ashbhandare · 2019-11-07T01:39:42Z

...classClassification/ImageClassification/LearningRateSchedulingCifarResnetTransferLearning.cs

+            // the same transforms as above.
+            IEnumerable<ImageData> test_images = LoadImagesFromDirectory(
+                folder: fullImagesetFolderPathTest, useFolderNameAsLabel: true);
+            IDataView testDataset = mlContext.Data.


IDataView testDataset = mlContext.Data. [](start = 12, length = 39)

new line #Resolved

ashbhandare · 2019-11-07T01:40:04Z

...classClassification/ImageClassification/LearningRateSchedulingCifarResnetTransferLearning.cs

+                folder: fullImagesetFolderPathTest, useFolderNameAsLabel: true);
+            IDataView testDataset = mlContext.Data.
+                LoadFromEnumerable(test_images);
+            testDataset = mlContext.Transforms.Conversion


testDataset = mlContext.Transforms.Conversion [](start = 12, length = 45)

newline #Resolved

ashbhandare · 2019-11-07T01:46:08Z

.../Dynamic/Trainers/MulticlassClassification/ImageClassification/ImageClassificationDefault.cs

@@ -119,37 +127,43 @@ private static void MlContext_Log(object sender, LoggingEventArgs e)
            }


add output from here to comments #Resolved

ashbhandare · 2019-11-07T01:46:31Z

...rs/MulticlassClassification/ImageClassification/ResnetV2101TransferLearningTrainTestSplit.cs

+                LearningRate = 0.01f,
+                MetricsCallback = (metrics) => Console.WriteLine(metrics),
+                ValidationSet = testDataset
+            };


null out ES #Resolved

codemzs

Remove duplicate lines from project file. (dotnet#4439) Image Classification Sample cleanup (dotnet#4380) * Aligned line ending to 85 char limit * Cleaned up Early Stopping sample * Added comments to ResnetV2101TransferLearningTrainTestSplit * Added comments to ResnetV2101TransferLearningEarlyStopping * Add comments to ImageClassificationDefault sample * Added comments to LearningRateSchedulingCifarResnetTransferLearning sample * added '.', formatted output. * fixed sample after rebase * fix ealy stopping sample * fix GetAbsolutePath function for samples. * remove unnecessary try catch block from samples. * review comments. Increment stable API version and add new stable packages to the list. (dotnet#4453) * Increment stable API version and add new stable packages to the list. * Increment stable API version and add new stable packages to the list. also update template file format template file fix error in template fix condition in template file fix template format error fix template parameter error fix on template, seems corrupted when copy from github fix placeholder fix template error, encoding fix template fix in template file set system env take comments and make adjustment

ashbhandare marked this pull request as ready for review October 25, 2019 22:30

ashbhandare requested a review from a team as a code owner October 25, 2019 22:30

ashbhandare force-pushed the SampleCleanup branch from d08871c to 3f529c8 Compare October 25, 2019 22:43

codemzs reviewed Oct 25, 2019

View reviewed changes

ashbhandare force-pushed the SampleCleanup branch 2 times, most recently from 25c17f6 to 7a2c421 Compare October 28, 2019 17:06

ashbhandare force-pushed the SampleCleanup branch from e90f6ba to 3f94d39 Compare November 4, 2019 21:26

ashbhandare added 9 commits November 6, 2019 16:45

Aligned line ending to 85 char limit

a3e044d

Cleaned up Early Stopping sample

217baf0

Added comments to ResnetV2101TransferLearningTrainTestSplit

471795c

Added comments to ResnetV2101TransferLearningEarlyStopping

cdde24a

Add comments to ImageClassificationDefault sample

2a24847

Added comments to LearningRateSchedulingCifarResnetTransferLearning s…

68e8246

…ample

added '.', formatted output.

8ef1da7

fixed sample after rebase

53c607a

fix ealy stopping sample

80d557b

ashbhandare force-pushed the SampleCleanup branch from 3f94d39 to 80d557b Compare November 7, 2019 00:45

ashbhandare added 2 commits November 6, 2019 16:48

fix GetAbsolutePath function for samples.

6297d37

remove unnecessary try catch block from samples.

1e11eb5

ashbhandare commented Nov 7, 2019

View reviewed changes

review comments.

d7eb1a1

codemzs approved these changes Nov 7, 2019

View reviewed changes

codemzs merged commit 0b9308e into dotnet:master Nov 7, 2019

ashbhandare deleted the SampleCleanup branch November 7, 2019 21:47

ghost locked as resolved and limited conversation to collaborators Mar 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Image Classification Sample cleanup #4380

Image Classification Sample cleanup #4380

ashbhandare commented Oct 24, 2019

codecov bot commented Oct 24, 2019 •

edited

Loading

codemzs Oct 25, 2019 •

edited by ashbhandare

Loading

codemzs Oct 25, 2019 •

edited by ashbhandare

Loading

codemzs Oct 25, 2019 •

edited by ashbhandare

Loading

codemzs Oct 25, 2019 •

edited by ashbhandare

Loading

ashbhandare Nov 7, 2019 •

edited

Loading

ashbhandare Nov 7, 2019 •

edited

Loading

ashbhandare Nov 7, 2019 •

edited

Loading

ashbhandare Nov 7, 2019 •

edited

Loading

ashbhandare Nov 7, 2019 •

edited

Loading

codemzs left a comment

		@@ -119,37 +127,43 @@ private static void MlContext_Log(object sender, LoggingEventArgs e)
		}

Image Classification Sample cleanup #4380

Image Classification Sample cleanup #4380

Conversation

ashbhandare commented Oct 24, 2019

codecov bot commented Oct 24, 2019 • edited Loading

Codecov Report

codemzs Oct 25, 2019 • edited by ashbhandare Loading

Choose a reason for hiding this comment

codemzs Oct 25, 2019 • edited by ashbhandare Loading

Choose a reason for hiding this comment

codemzs Oct 25, 2019 • edited by ashbhandare Loading

Choose a reason for hiding this comment

codemzs Oct 25, 2019 • edited by ashbhandare Loading

Choose a reason for hiding this comment

ashbhandare Nov 7, 2019 • edited Loading

Choose a reason for hiding this comment

ashbhandare Nov 7, 2019 • edited Loading

Choose a reason for hiding this comment

ashbhandare Nov 7, 2019 • edited Loading

Choose a reason for hiding this comment

ashbhandare Nov 7, 2019 • edited Loading

Choose a reason for hiding this comment

ashbhandare Nov 7, 2019 • edited Loading

Choose a reason for hiding this comment

codemzs left a comment

Choose a reason for hiding this comment

codecov bot commented Oct 24, 2019 •

edited

Loading

codemzs Oct 25, 2019 •

edited by ashbhandare

Loading

codemzs Oct 25, 2019 •

edited by ashbhandare

Loading

codemzs Oct 25, 2019 •

edited by ashbhandare

Loading

codemzs Oct 25, 2019 •

edited by ashbhandare

Loading

ashbhandare Nov 7, 2019 •

edited

Loading

ashbhandare Nov 7, 2019 •

edited

Loading

ashbhandare Nov 7, 2019 •

edited

Loading

ashbhandare Nov 7, 2019 •

edited

Loading

ashbhandare Nov 7, 2019 •

edited

Loading