Lockdown HAL Project #2497

Ivanidzo4ka · 2019-02-11T19:09:00Z

Ivanidzo4ka · 2019-02-11T19:09:31Z

src/Microsoft.ML.HalLearners/HalLearnersCatalog.cs

-        /// <param name="weights">The weights column.</param>
+        /// <param name="labelColumn">The name of label column.</param>
+        /// <param name="featureColumn">The name of feature column.</param>
+        /// <param name="weightColumn">The name of weight column.</param>


weight [](start = 51, length = 6)

optional! #Closed

Ivanidzo4ka · 2019-02-11T19:09:55Z

src/Microsoft.ML.HalLearners/HalLearnersCatalog.cs

+        /// <example>
+        /// <format type="text/markdown">
+        /// <![CDATA[
+        /// [!code-csharp[ScoreTensorFlowModel](~/../docs/samples/docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/Regression/OrdinaryLeastSquaresWithOptions.cs)]


ScoreTensorFlowModel [](start = 26, length = 20)

oops #Closed

codecov · 2019-02-12T01:54:02Z

Codecov Report

❗ No coverage uploaded for pull request base (master@b2587fa). Click here to learn what that means.
The diff coverage is 86.95%.

@@            Coverage Diff            @@
##             master    #2497   +/-   ##
=========================================
  Coverage          ?   71.45%           
=========================================
  Files             ?      801           
  Lines             ?   141840           
  Branches          ?    16139           
=========================================
  Hits              ?   101352           
  Misses            ?    36019           
  Partials          ?     4469

Flag	Coverage Δ
#Debug	`71.45% <86.95%> (?)`
#production	`67.74% <81.25%> (?)`
#test	`85.52% <100%> (?)`

zeahmed · 2019-02-12T03:02:15Z

...rosoft.ML.Samples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescent.cs

+
+            // Leave out 10% of data for testing
+            var (trainData, testData) = mlContext.BinaryClassification.TrainTestSplit(data, testFraction: 0.1);
+            // Get DefaultBinaryPipine for adult dataset and append SymSGD to it.


DefaultBinaryPipine [](start = 19, length = 19)

typo. #Closed

zeahmed · 2019-02-12T03:05:17Z

src/Microsoft.ML.SamplesUtils/SamplesDatasetUtils.cs

+            return reader.Read(dataFile);
+        }
+
+        public static IEstimator<ITransformer> DefaultBinaryPipeline(MLContext mlContext)


DefaultBinaryPipeline [](start = 47, length = 21)

DefaultBinaryPipelineForAdultDataSet???

Its confusing, unless I see this method definition here. The usage seems like we are creating binary pipeline in general. #Closed

zeahmed · 2019-02-12T03:07:04Z

src/Microsoft.ML.SamplesUtils/SamplesDatasetUtils.cs

+                   "occupation", "relationship", "ethnicity", "native-country", "age", "education-num",
+                   "capital-gain", "capital-loss", "hours-per-week"))
+               // Min-max normalized all the features
+               .Append(mlContext.Transforms.Normalize("Features"));


(mlContext.Transforms.Normalize("Features") [](start = 22, length = 43)

There is no need to normalize OneHotEncoded variables as they already range between 0-1. Here we should only normalize numeric features. #Closed

zeahmed · 2019-02-12T03:08:14Z

...amples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescentWithOptions.cs

+
+            // Leave out 10% of data for testing
+            var (trainData, testData) = mlContext.BinaryClassification.TrainTestSplit(data, testFraction: 0.1);
+            // Get DefaultBinaryPipine for adult dataset and append SymSGD to it.


DefaultBinaryPipine [](start = 19, length = 19)

typo. #Closed

zeahmed · 2019-02-12T03:10:30Z

docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/Regression/OrdinaryLeastSquares.cs

+        public static void Example()
+        {
+            // Downloading a regression dataset from github.com/dotnet/machinelearning
+            // this will create a housing.txt file in the filsystem this code will run


this code will run [](start = 68, length = 18)

This sentence does not convey anything? #Closed

I feel we should have some sort of template kind of thing for all the code and comments because most of the code and comments are shared across many different samples. What do you say? maybe tt template?
cc: @sfilipi, @rogancarr, @shmoradims

In reply to: 255788213 [](ancestors = 255788213)

sfilipi · 2019-02-12T14:34:38Z

...rosoft.ML.Samples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescent.cs

+{
+    public sealed class SymbolicStochasticGradientDescent
+    {
+        /// <summary>


/// <su [](start = 8, length = 7)

no need for xml style comments #Resolved

sfilipi · 2019-02-12T14:35:02Z

...rosoft.ML.Samples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescent.cs

+            // In this examples we will use the adult income dataset. The goal is to predict
+            // if a person's income is above $50K or not, based on different pieces of information about that person.
+            // For more details about this dataset, please see https://archive.ics.uci.edu/ml/datasets/adult
+


i'd move this comment above the method. #Resolved

sfilipi · 2019-02-12T14:38:50Z

...rosoft.ML.Samples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescent.cs

+namespace Microsoft.ML.Samples.Dynamic
+{
+    public sealed class SymbolicStochasticGradientDescent
+    {


cc @shmoradims, @jwood803 so you don't work on the same. #Closed

sfilipi · 2019-02-12T14:42:30Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Projection/VectorWhiten.cs

+
+            printHelper(nameof(SamplesUtils.DatasetUtils.SampleVectorOfNumbersData.Features), whitening);
+
+            // Features column obtained post-transformation.


column [](start = 24, length = 6)

i think we should come up with a convention to print vector columns differently from columns of the dataset.
I'll take a look at what .Net does.
cc @shmoradims, @rogancarr

maybe put in [ ]

//[ -0.394 -0.318 -0.243 -0.168 0.209 0.358 0.433 0.589 0.873 2.047 ]

I like the suggestion of adding [...]

In reply to: 255982945 [](ancestors = 255982945)

artidoro · 2019-02-13T09:13:14Z

...amples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescentWithOptions.cs

+        /// <summary>
+        /// This example require installation of addition nuget package <a href="https://www.nuget.org/packages/Microsoft.ML.HalLearners/">Microsoft.ML.HalLearners</a>
+        /// </summary>
+        public static void Example()


Xml #Resolved

artidoro · 2019-02-13T09:13:37Z

...rosoft.ML.Samples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescent.cs

+    public sealed class SymbolicStochasticGradientDescent
+    {
+        /// <summary>
+        /// This example require installation of addition nuget package <a href="https://www.nuget.org/packages/Microsoft.ML.HalLearners/">Microsoft.ML.HalLearners</a>


Could you replace "requires" instead of "require"? #Resolved

artidoro · 2019-02-13T09:22:11Z

Should we make this a normal integer, with default value? Otherwise could you explain what the default is in the xml comment? #Resolved

Refers to: src/Microsoft.ML.HalLearners/SymSgdClassificationTrainer.cs:52 in 1527369. [](commit_id = 1527369, deletion_comment = False)

artidoro · 2019-02-13T09:23:42Z

I see this applies to quite a few instances in this class, and probably also in many others.
Would you suggest to file this as an issue and work on it in a separate PR?

In reply to: 463122337 [](ancestors = 463122337)

Refers to: src/Microsoft.ML.HalLearners/SymSgdClassificationTrainer.cs:52 in 1527369. [](commit_id = 1527369, deletion_comment = False)

Ivanidzo4ka · 2019-02-15T18:33:27Z

Think about this nullable values as something which can be tuned by algorithm. If it's specified we will use it, and if we not, we will run code to pick best possible value.

In reply to: 463122866 [](ancestors = 463122866,463122337)

Refers to: src/Microsoft.ML.HalLearners/SymSgdClassificationTrainer.cs:52 in 1527369. [](commit_id = 1527369, deletion_comment = False)

artidoro · 2019-02-15T18:39:27Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Projection/VectorWhiten.cs

+        /// <summary>
+        /// This example require installation of addition nuget package <a href="https://www.nuget.org/packages/Microsoft.ML.HalLearners/">Microsoft.ML.HalLearners</a>
+        /// </summary>
+        public static void Example()


Can you remove the xml? #Resolved

artidoro · 2019-02-15T18:41:44Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Projection/VectorWhitenWithColumnInfo.cs

+        /// <summary>
+        /// This example require installation of addition nuget package <a href="https://www.nuget.org/packages/Microsoft.ML.HalLearners/">Microsoft.ML.HalLearners</a>
+        /// </summary>
+        public static void Example()


Here too, could you remove the xml? #Resolved

artidoro · 2019-02-15T18:42:51Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Projection/VectorWhitenWithColumnInfo.cs

+            printHelper(nameof(SamplesUtils.DatasetUtils.SampleVectorOfNumbersData.Features), whitening);
+
+            // Features column obtained post-transformation.
+            // -0.979  0.867  1.449  1.236


0 [](start = 23, length = 1)

I think this is a vector column, if so, could you use senja's suggestion of printing the vector inside brackets: [....]

artidoro · 2019-02-15T18:45:22Z

src/Microsoft.ML.HalLearners/HalLearnersCatalog.cs

-        /// <param name="labelColumn">The labelColumn column.</param>
-        /// <param name="featureColumn">The features column.</param>
-        /// <param name="weights">The weights column.</param>
+        /// <param name="labelColumn">The name of label column.</param>


label [](start = 50, length = 5)

Could you add 'the' after of in these xml comments?
I think it sounds better like this: "The name of the label column.", and I think that's how we have done it in other xml doc. #Resolved

artidoro · 2019-02-15T18:46:42Z

src/Microsoft.ML.HalLearners/HalLearnersCatalog.cs

@@ -60,9 +74,18 @@ public static class HalLearnersCatalog
        /// <param name="catalog">The <see cref="BinaryClassificationCatalog"/>.</param>
        /// <param name="labelColumn">The labelColumn column.</param>


labelColumn [](start = 42, length = 11)

Could you make the change you did above here too? Specifying that this is the name of the label column, instead of just saying label column.
#Resolved

artidoro · 2019-02-15T18:47:48Z

src/Microsoft.ML.HalLearners/OlsLinearRegression.cs

@@ -87,9 +89,6 @@ internal OlsLinearRegressionTrainer(IHostEnvironment env, Options options)
        protected override RegressionPredictionTransformer<OlsLinearRegressionModelParameters> MakeTransformer(OlsLinearRegressionModelParameters model, DataViewSchema trainSchema)
             => new RegressionPredictionTransformer<OlsLinearRegressionModelParameters>(Host, model, trainSchema, FeatureColumn.Name);

-        public RegressionPredictionTransformer<OlsLinearRegressionModelParameters> Train(IDataView trainData, IPredictor initialPredictor = null)
-            => TrainTransformer(trainData, initPredictor: initialPredictor);


Why are we removing this? #Resolved

Because it's not used at all.

In reply to: 257348853 [](ancestors = 257348853)

artidoro · 2019-02-15T18:53:53Z

src/Microsoft.ML.HalLearners/OlsLinearRegression.cs

-                    score = float.MaxValue;
-                editor.Values[i] = score;
-            }
-            weights = editor.Commit();


It seems that this is lost in the base method? Do we need it? #Resolved

See #1850

In reply to: 257350745 [](ancestors = 257350745)

zeahmed · 2019-02-15T19:01:25Z

...rosoft.ML.Samples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescent.cs

+            // Setting the seed to a fixed number in this examples to make outputs deterministic.
+            var mlContext = new MLContext(seed: 0);
+
+            // Download and featurize the dataset


Download and featurize the dataset [](start = 15, length = 34)

I saw @rogancarr comment regarding ending every comment with period. #Closed

zeahmed · 2019-02-15T19:03:59Z

...rosoft.ML.Samples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescent.cs

@@ -0,0 +1,39 @@
+namespace Microsoft.ML.Samples.Dynamic
+{
+    public sealed class SymbolicStochasticGradientDescent


SymbolicStochasticGradientDescent [](start = 24, length = 33)

We also need to decide the fate of the classes in sample project. I see some of the classes are static some are sealed.
cc: @rogancarr, @sfilipi, @shmoradims #Resolved

artidoro · 2019-02-15T19:04:43Z

src/Microsoft.ML.HalLearners/SymSgdClassificationTrainer.cs

+        /// Continue to train <paramref name="initialPredictor"/> on <paramref name="trainData"/> and produce calibrated binary linear model.
+        /// </summary>
+        /// <param name="trainData">The training data set.</param>
+        /// <param name="initialPredictor">Accepts <see cref="LinearBinaryModelParameters"/> to continue training of this model</param>
        public BinaryPredictionTransformer<TPredictor> Train(IDataView trainData, TPredictor initialPredictor = null)


IDataView [](start = 61, length = 9)

You might need to merge with master. I should have added a doc for this. #Resolved

zeahmed · 2019-02-15T19:05:29Z

docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/Regression/OrdinaryLeastSquares.cs

+            // Create a new ML context, for ML.NET operations. It can be used for exception tracking and logging, 
+            // as well as the source of randomness.
+            var mlContext = new MLContext(seed: 3);
+            // Creating a data reader, based on the format of the data


newline please. #Closed

zeahmed · 2019-02-15T19:09:12Z

...rosoft.ML.Samples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescent.cs

+    public sealed class SymbolicStochasticGradientDescent
+    {
+
+        // This example requires installation of addition nuget package <a href="https://www.nuget.org/packages/Microsoft.ML.HalLearners/">Microsoft.ML.HalLearners</a>


addition [](start = 49, length = 8)

addition -> additional

correct it everywhere. #Closed

zeahmed · 2019-02-15T19:10:03Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Projection/VectorWhiten.cs

+                Console.WriteLine($"{colName} column obtained post-transformation.");
+                foreach (var row in column)
+                    Console.WriteLine($"{string.Join(" ", row.DenseValues().Select(x => x.ToString("f3")))} ");
+            };


Can it go to SamplesUtils? #Closed

Eventually. Not in this PR

In reply to: 257356248 [](ancestors = 257356248)

ok.

In reply to: 257356644 [](ancestors = 257356644,257356248)

artidoro

zeahmed · 2019-02-15T19:24:40Z

...rosoft.ML.Samples/Dynamic/Trainers/BinaryClassification/SymbolicStochasticGradientDescent.cs

+{
+    public static class SymbolicStochasticGradientDescent
+    {
+        // This example requires installation of additional nuget package <a href="https://www.nuget.org/packages/Microsoft.ML.HalLearners/">Microsoft.ML.HalLearners</a>


[](start = 165, length = 4)

period here as well.

zeahmed

Overall it LGTM. Thanks for doing it.

Ivan Matantsev added 2 commits February 11, 2019 10:59

Lockdown Hal project

7806211

Merge remote-tracking branch 'upstream/master' into Ivanidze/LockdownHAL

cc5cf79

Ivanidzo4ka commented Feb 11, 2019

View reviewed changes

Ivanidzo4ka requested review from sfilipi, shmoradims, zeahmed and rogancarr February 11, 2019 19:20

small updates

1527369

zeahmed reviewed Feb 12, 2019

View reviewed changes

sfilipi reviewed Feb 12, 2019

View reviewed changes

artidoro reviewed Feb 13, 2019

View reviewed changes

Ivan Matantsev added 6 commits February 13, 2019 11:11

Merge branch 'master' into Ivanidze/LockdownHAL

a78fccd

merge with master

d70c352

Merge with master

065207f

Merge with master

002af90

merge with master

ea9d1d2

Fix merge conflicts

140d69d

artidoro reviewed Feb 15, 2019

View reviewed changes

zeahmed reviewed Feb 15, 2019

View reviewed changes

artidoro reviewed Feb 15, 2019

View reviewed changes

zeahmed reviewed Feb 15, 2019

View reviewed changes

Ivan Matantsev added 2 commits February 15, 2019 11:10

Address comments

5b19660

Phrasing

e2634e1

artidoro approved these changes Feb 15, 2019

View reviewed changes

zeahmed reviewed Feb 15, 2019

View reviewed changes

zeahmed approved these changes Feb 15, 2019

View reviewed changes

Make it build

b67de2a

Ivanidzo4ka merged commit 78b161b into dotnet:master Feb 15, 2019

rogancarr mentioned this pull request Apr 15, 2019

OLS FeatureWeights are not the model weights #1850

Closed

ghost locked as resolved and limited conversation to collaborators Mar 24, 2022


		printHelper(nameof(SamplesUtils.DatasetUtils.SampleVectorOfNumbersData.Features), whitening);

		// Features column obtained post-transformation.

		@@ -60,9 +74,18 @@ public static class HalLearnersCatalog
		/// <param name="catalog">The <see cref="BinaryClassificationCatalog"/>.</param>
		/// <param name="labelColumn">The labelColumn column.</param>

Lockdown HAL Project #2497

Lockdown HAL Project #2497

Conversation

Ivanidzo4ka commented Feb 11, 2019

Ivanidzo4ka Feb 11, 2019 • edited Loading

Choose a reason for hiding this comment

Ivanidzo4ka Feb 11, 2019 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Feb 12, 2019 • edited Loading

Codecov Report

zeahmed Feb 12, 2019 • edited Loading

Choose a reason for hiding this comment

zeahmed Feb 12, 2019 • edited Loading

Choose a reason for hiding this comment

zeahmed Feb 12, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

zeahmed Feb 12, 2019 • edited Loading

Choose a reason for hiding this comment

zeahmed Feb 12, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

zeahmed Feb 12, 2019 • edited Loading

Choose a reason for hiding this comment

sfilipi Feb 12, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

sfilipi Feb 12, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

sfilipi Feb 12, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

artidoro Feb 13, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

artidoro Feb 13, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

artidoro commented Feb 13, 2019 • edited by Ivanidzo4ka Loading

artidoro commented Feb 13, 2019

Ivanidzo4ka commented Feb 15, 2019

artidoro Feb 15, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

artidoro Feb 15, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

artidoro Feb 15, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

artidoro Feb 15, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

artidoro Feb 15, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

artidoro Feb 15, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zeahmed Feb 15, 2019 • edited Loading

Choose a reason for hiding this comment

zeahmed Feb 15, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

artidoro Feb 15, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

zeahmed Feb 15, 2019 • edited Loading

Choose a reason for hiding this comment

zeahmed Feb 15, 2019 • edited Loading

Choose a reason for hiding this comment

zeahmed Feb 15, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

artidoro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zeahmed left a comment

Choose a reason for hiding this comment

Ivanidzo4ka Feb 11, 2019 •

edited

Loading

Ivanidzo4ka Feb 11, 2019 •

edited

Loading

codecov bot commented Feb 12, 2019 •

edited

Loading

zeahmed Feb 12, 2019 •

edited

Loading

zeahmed Feb 12, 2019 •

edited

Loading

zeahmed Feb 12, 2019 •

edited by Ivanidzo4ka

Loading

zeahmed Feb 12, 2019 •

edited

Loading

zeahmed Feb 12, 2019 •

edited by Ivanidzo4ka

Loading

zeahmed Feb 12, 2019 •

edited

Loading

sfilipi Feb 12, 2019 •

edited by Ivanidzo4ka

Loading

sfilipi Feb 12, 2019 •

edited by Ivanidzo4ka

Loading

sfilipi Feb 12, 2019 •

edited by Ivanidzo4ka

Loading

artidoro Feb 13, 2019 •

edited by Ivanidzo4ka

Loading

artidoro Feb 13, 2019 •

edited by Ivanidzo4ka

Loading

artidoro commented Feb 13, 2019 •

edited by Ivanidzo4ka

Loading

artidoro Feb 15, 2019 •

edited by Ivanidzo4ka

Loading

artidoro Feb 15, 2019 •

edited by Ivanidzo4ka

Loading

artidoro Feb 15, 2019 •

edited by Ivanidzo4ka

Loading

artidoro Feb 15, 2019 •

edited by Ivanidzo4ka

Loading

artidoro Feb 15, 2019 •

edited by Ivanidzo4ka

Loading

artidoro Feb 15, 2019 •

edited by Ivanidzo4ka

Loading

zeahmed Feb 15, 2019 •

edited

Loading

zeahmed Feb 15, 2019 •

edited by Ivanidzo4ka

Loading

artidoro Feb 15, 2019 •

edited by Ivanidzo4ka

Loading

zeahmed Feb 15, 2019 •

edited

Loading

zeahmed Feb 15, 2019 •

edited

Loading

zeahmed Feb 15, 2019 •

edited

Loading