Samples for categorical transform estimators #3179

abgoswam · 2019-04-02T21:01:45Z

Towards #1209

The PR makes the following changes

Adds sample for the OneHotHashEncoding transform estimator.
Updated sample for the OneHotEncoding transform estimator.

codecov · 2019-04-02T21:39:42Z

Codecov Report

Merging #3179 into master will increase coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3179      +/-   ##
==========================================
+ Coverage   72.54%   72.58%   +0.04%     
==========================================
  Files         807      807              
  Lines      144774   144956     +182     
  Branches    16208    16212       +4     
==========================================
+ Hits       105021   105212     +191     
+ Misses      35339    35326      -13     
- Partials     4414     4418       +4

Flag	Coverage Δ
#Debug	`72.58% <ø> (+0.04%)`	⬆️
#production	`68.14% <ø> (+0.01%)`	⬆️
#test	`88.88% <ø> (+0.05%)`	⬆️

Impacted Files	Coverage Δ
src/Microsoft.ML.Transforms/CategoricalCatalog.cs	`68.42% <ø> (ø)`	⬆️
src/Microsoft.ML.DataView/KeyDataViewType.cs	`74.57% <0%> (-3.76%)`	⬇️
src/Microsoft.ML.Maml/MAML.cs	`24.75% <0%> (-1.46%)`	⬇️
src/Microsoft.ML.Transforms/Text/LdaTransform.cs	`89.26% <0%> (-0.63%)`	⬇️
test/Microsoft.ML.Tests/ImagesTests.cs	`98.69% <0%> (-0.13%)`	⬇️
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs	`100% <0%> (ø)`	⬆️
...ML.Data/Transforms/ConversionsExtensionsCatalog.cs	`44.87% <0%> (ø)`	⬆️
src/Microsoft.ML.Transforms/Text/TextCatalog.cs	`41.66% <0%> (ø)`	⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs	`84.9% <0%> (+0.2%)`	⬆️
...rosoft.ML.ImageAnalytics/VectorToImageTransform.cs	`76.77% <0%> (+4.53%)`	⬆️
... and 3 more

sfilipi · 2019-04-03T15:50:41Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Categorical/OneHotEncoding.cs


            // A pipeline for one hot encoding the Education column.
-            var bagPipeline = mlContext.Transforms.Categorical.OneHotEncoding("EducationOneHotEncoded", "Education", OutputKind.Bag);


", OutputKind.Bag); [](start = 114, length = 19)

I would leave it, so that it makes sense why we call it bagPipeline. #Closed

am using the default (which is uses Indicator , not bagging) .. also renamed it to pipeline

In reply to: 271811206 [](ancestors = 271811206)

sfilipi · 2019-04-03T15:51:44Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Categorical/OneHotEncoding.cs

+            // 1 0 0                   0 1
+            // 0 1 0                   1 0
+            // 0 1 0                   0 1
+            // 0 0 1                   1 0
        }


make it a separate example, because the multi-output is a different API. #Closed

made separate example for multi input

In reply to: 271811670 [](ancestors = 271811670)

sfilipi · 2019-04-03T15:52:09Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Categorical/OneHotHashEncoding.cs

+            // 0 0 0 0 1 0 0 0                 0 0 0 0 0 0 0 1
+            // 0 0 0 0 1 0 0 0                 1 0 0 0 0 0 0 0
+            // 0 0 0 0 0 0 0 1                 0 0 0 0 0 0 0 1
+        }


separate example. #Resolved

created separate example for multi column

In reply to: 271811872 [](ancestors = 271811872)

shmoradims · 2019-04-04T00:25:00Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Categorical/OneHotEncoding.cs

        private class DataPoint
        {
            public float Label { get; set; }

            public string Education { get; set; }
+
+            public string ZipCode { get; set; }


public string ZipCode { get; set; } [](start = 11, length = 36)

please remove since it's not used here #Resolved

shmoradims · 2019-04-04T00:25:09Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Categorical/OneHotHashEncoding.cs

+
+            public string Education { get; set; }
+
+            public string ZipCode { get; set; }


public string ZipCode { get; set; } [](start = 11, length = 36)

please remove since it's not used here #Resolved

shmoradims · 2019-04-04T00:26:40Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Categorical/OneHotEncoding.cs

+                    Console.Write($"{row[i]}\t");
+                Console.WriteLine();
+            }
+        }
        private class DataPoint
        {
            public float Label { get; set; }


public float Label { get; set; } [](start = 12, length = 32)

please remove #Resolved

shmoradims

shmoradims · 2019-04-04T17:44:41Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Categorical/OneHotEncoding.cs

@@ -1,6 +1,5 @@
 using System;
 using System.Collections.Generic;
-using System.Linq;
 using Microsoft.ML.Data;
 using static Microsoft.ML.Transforms.OneHotEncodingEstimator;


what is this for? can we remove it?
#Resolved

this is for the OutputKind.Key parameter that we use in the example below

In reply to: 272292647 [](ancestors = 272292647)

shmoradims · 2019-04-04T19:01:06Z

namespace Microsoft.ML.Samples.Dynamic

let's also drop Microsoft.ML prefix while we're at it. #Resolved

Refers to: docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Categorical/OneHotEncoding.cs:6 in 4cd6397. [](commit_id = 4cd6397, deletion_comment = False)

sfilipi

* categorical transform estimators * review comments * fix review comments * modify samples namespace

abgoswam added 2 commits April 2, 2019 20:44

categorical transform estimators

e21dc3f

Merge branch 'master' into abgoswam/categorical_samples

d646f2e

abgoswam requested review from sfilipi, Ivanidzo4ka and shmoradims April 2, 2019 21:01

sfilipi mentioned this pull request Apr 2, 2019

API reference - Samples for Transforms #1209

Closed

sfilipi reviewed Apr 3, 2019

View reviewed changes

review comments

a4f9ee8

shmoradims reviewed Apr 4, 2019

View reviewed changes

fix review comments

4cd6397

shmoradims approved these changes Apr 4, 2019

View reviewed changes

shmoradims reviewed Apr 4, 2019

View reviewed changes

modify samples namespace

ea7a01e

abgoswam requested a review from rogancarr April 4, 2019 21:27

sfilipi approved these changes Apr 4, 2019

View reviewed changes

abgoswam merged commit 9d79ab3 into dotnet:master Apr 4, 2019

abgoswam added a commit to abgoswam/machinelearning that referenced this pull request Apr 5, 2019

Samples for categorical transform estimators (dotnet#3179)

b9ee885

* categorical transform estimators * review comments * fix review comments * modify samples namespace

abgoswam mentioned this pull request Apr 5, 2019

Cherry pick for samples {Image, Categorical, FeatureSelection} #3221

Closed

ghost locked as resolved and limited conversation to collaborators Mar 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Samples for categorical transform estimators #3179

Samples for categorical transform estimators #3179

abgoswam commented Apr 2, 2019

codecov bot commented Apr 2, 2019 •

edited

Loading

sfilipi Apr 3, 2019 •

edited

Loading

abgoswam Apr 4, 2019

sfilipi Apr 3, 2019 •

edited

Loading

abgoswam Apr 4, 2019

sfilipi Apr 3, 2019 •

edited by abgoswam

Loading

abgoswam Apr 4, 2019

shmoradims Apr 4, 2019 •

edited by abgoswam

Loading

shmoradims Apr 4, 2019 •

edited by abgoswam

Loading

shmoradims Apr 4, 2019 •

edited by abgoswam

Loading

shmoradims left a comment

shmoradims Apr 4, 2019 •

edited by abgoswam

Loading

abgoswam Apr 4, 2019

shmoradims commented Apr 4, 2019 •

edited by abgoswam

Loading

sfilipi left a comment


		// A pipeline for one hot encoding the Education column.
		var bagPipeline = mlContext.Transforms.Categorical.OneHotEncoding("EducationOneHotEncoded", "Education", OutputKind.Bag);


		public string Education { get; set; }

		public string ZipCode { get; set; }

Samples for categorical transform estimators #3179

Samples for categorical transform estimators #3179

Conversation

abgoswam commented Apr 2, 2019

codecov bot commented Apr 2, 2019 • edited Loading

Codecov Report

sfilipi Apr 3, 2019 • edited Loading

Choose a reason for hiding this comment

abgoswam Apr 4, 2019

Choose a reason for hiding this comment

sfilipi Apr 3, 2019 • edited Loading

Choose a reason for hiding this comment

abgoswam Apr 4, 2019

Choose a reason for hiding this comment

sfilipi Apr 3, 2019 • edited by abgoswam Loading

Choose a reason for hiding this comment

abgoswam Apr 4, 2019

Choose a reason for hiding this comment

shmoradims Apr 4, 2019 • edited by abgoswam Loading

Choose a reason for hiding this comment

shmoradims Apr 4, 2019 • edited by abgoswam Loading

Choose a reason for hiding this comment

shmoradims Apr 4, 2019 • edited by abgoswam Loading

Choose a reason for hiding this comment

shmoradims left a comment

Choose a reason for hiding this comment

shmoradims Apr 4, 2019 • edited by abgoswam Loading

Choose a reason for hiding this comment

abgoswam Apr 4, 2019

Choose a reason for hiding this comment

shmoradims commented Apr 4, 2019 • edited by abgoswam Loading

sfilipi left a comment

Choose a reason for hiding this comment

codecov bot commented Apr 2, 2019 •

edited

Loading

sfilipi Apr 3, 2019 •

edited

Loading

sfilipi Apr 3, 2019 •

edited

Loading

sfilipi Apr 3, 2019 •

edited by abgoswam

Loading

shmoradims Apr 4, 2019 •

edited by abgoswam

Loading

shmoradims Apr 4, 2019 •

edited by abgoswam

Loading

shmoradims Apr 4, 2019 •

edited by abgoswam

Loading

shmoradims Apr 4, 2019 •

edited by abgoswam

Loading

shmoradims commented Apr 4, 2019 •

edited by abgoswam

Loading