Key to binary samples for documentation #3211

sfilipi · 2019-04-05T06:18:48Z

Towards #1209
This adds the last batch of samples for the Conversions catalog.
The samples are KeyToVector and KeyToBinaryVector

sfilipi · 2019-04-05T06:22:48Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Conversion/MapKeyToVectorMultiColumn.cs

+        /// This example demonstrates the use of MapKeyToVector by mapping keys to floats[] for multiple columns at once. 
+        /// Because the ML.NET KeyType maps the missing value to zero, counting of uints starts at 1, so the values
+        /// converted to KeyTypes will appear skewed by one. See  https://github.com/dotnet/machinelearning/issues/3072
+        public static void Example()


i have tried to add explanations, because of issue 3072.
I think we should change the mapping of the missing value to 0... #WontFix

codecov · 2019-04-05T06:57:10Z

Codecov Report

Merging #3211 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3211      +/-   ##
==========================================
+ Coverage   72.62%   72.62%   +<.01%     
==========================================
  Files         807      807              
  Lines      145080   145080              
  Branches    16213    16213              
==========================================
+ Hits       105369   105370       +1     
  Misses      35294    35294              
+ Partials     4417     4416       -1

Flag	Coverage Δ
#Debug	`72.62% <ø> (ø)`	⬆️
#production	`68.17% <ø> (ø)`	⬆️
#test	`88.92% <ø> (ø)`	⬆️

Impacted Files	Coverage Δ
src/Microsoft.ML.Data/Data/SchemaDefinition.cs	`69.87% <ø> (ø)`	⬆️
...ML.Data/Transforms/ConversionsExtensionsCatalog.cs	`64.07% <ø> (ø)`	⬆️
src/Microsoft.ML.Transforms/ConversionsCatalog.cs	`83.33% <ø> (ø)`	⬆️
...soft.ML.Transforms/Text/WordEmbeddingsExtractor.cs	`87.52% <0%> (-0.91%)`	⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs	`86.26% <0%> (+0.15%)`	⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs	`84.9% <0%> (+0.2%)`	⬆️
src/Microsoft.ML.Maml/MAML.cs	`26.21% <0%> (+1.45%)`	⬆️

sfilipi · 2019-04-05T15:51:59Z

Referencing #3072 since the previews on those samples illustrate what might be a source of confusion. #WontFix

wschin · 2019-04-06T00:11:51Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Conversion/MapKeyToVector.cs

+            // TransformedData obtained post-transformation.
+            //
+            // Timeframe          TimeframeVector    Category    CategoryVector
+            //  10              0,0,0,0,0,0,0,0,0,1       6          0,0,0,0,0


Is this correct? The first Timeframe was 9 in your declaration. If no transformer is applied to Timeframe, ML.NET should not touch its value? Is my understanding correct? Or I am missing something? #Resolved

Yeah.... this is why i logged 3072, and had a discussion on Friday post scrum. Gets easier to think of them as strings, not numbers.

the string "0", gets mapped to the first available ulong, 1. (The ulong 0 is reserved for missing key).
The string "1" gets mapped to the next available : 2.
On our KeyType system counting starts from 1. 0 is reserved.

In reply to: 272774353 [](ancestors = 272774353)

rogancarr · 2019-04-08T21:14:02Z

    internal static KeyToBinaryVectorMappingEstimator MapKeyToBinaryVector(this TransformsCatalog.ConversionTransforms catalog,

Link to Sample? #Resolved

Refers to: src/Microsoft.ML.Transforms/ConversionsCatalog.cs:23 in 9db4dfe. [](commit_id = 9db4dfe, deletion_comment = False)

rogancarr · 2019-04-08T21:15:46Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Conversion/MapKeyToVector.cs

+
+            Console.WriteLine($" Timeframe           TimeframeVector         Category    CategoryVector");
+            foreach (var featureRow in features)
+                Console.WriteLine($"{featureRow.Timeframe}\t\t\t{string.Join(',', featureRow.TimeframeVector)}\t\t\t{featureRow.Category}\t\t{string.Join(',', featureRow.CategoryVector)}");


($" [](start = 33, length = 3)

For this long string, consider doing the old-style "{0} {1}", thingOne, thingTwo etc.

As it is, it's hard to read. #Pending

removed the tabs instead.

In reply to: 273242076 [](ancestors = 273242076)

rogancarr

LGTM. Just a few small comments.

artidoro · 2019-04-08T22:28:59Z

docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/Conversion/MapKeyToVectorMultiColumn.cs

+
+            // Constructs the ML.net pipeline
+            var pipeline = mlContext.Transforms.Conversion.MapKeyToVector(
+                new[]{


new[]{ [](start = 16, length = 6)

nit: maybe keep this ("new[]{") on the previous line, or indent the next two. #Resolved

artidoro · 2019-04-08T22:42:54Z

src/Microsoft.ML.Data/Data/SchemaDefinition.cs

-        /// Marks member as <see cref="KeyDataViewType"/> and specifies <see cref="KeyDataViewType"/> cardinality.
+        /// Marks member as <see cref="KeyDataViewType"/>. The <paramref name="count"/> should be set to
+        /// one more than the maximum value for the keys (to account for missing values).
+        /// If the values are outside of the specified cardinality they will be mapped to the missing value representation: 0.
        /// </summary>


I think that the old description was correct, especially when the underlying values are not numbers. When that's the case, I think it might make more sense to think about cardinality instead of maximum value of keys. #Resolved

Maybe what you have written could be added as a note for the special case of integer values?

In reply to: 273266746 [](ancestors = 273266746)

artidoro

sfilipi · 2019-04-09T17:28:49Z

    internal static KeyToBinaryVectorMappingEstimator MapKeyToBinaryVector(this TransformsCatalog.ConversionTransforms catalog,

it is internal, it does have one yet.

In reply to: 481010536 [](ancestors = 481010536)

Refers to: src/Microsoft.ML.Transforms/ConversionsCatalog.cs:23 in 9db4dfe. [](commit_id = 9db4dfe, deletion_comment = False)

* adding samples for KeyToVector and KeyToBinaryVector * typo * Adding pointers to the KeyType. * PR review comments

sfilipi requested review from shmoradims, Ivanidzo4ka and zeahmed April 5, 2019 06:19

sfilipi added the documentation Related to documentation of ML.NET label Apr 5, 2019

sfilipi commented Apr 5, 2019

View reviewed changes

wschin reviewed Apr 6, 2019

View reviewed changes

rogancarr reviewed Apr 8, 2019

View reviewed changes

rogancarr approved these changes Apr 8, 2019

View reviewed changes

artidoro reviewed Apr 8, 2019

View reviewed changes

artidoro approved these changes Apr 8, 2019

View reviewed changes

sfilipi added 4 commits April 9, 2019 12:27

adding samples for KeyToVector and KeyToBinaryVector

bd2c3e1

typo

14582c0

Adding pointers to the KeyType.

8e6a295

PR review comments

e8bf494

sfilipi force-pushed the keyToBinary branch from 9db4dfe to e8bf494 Compare April 9, 2019 19:32

sfilipi merged commit ad99bc7 into dotnet:master Apr 9, 2019

sfilipi deleted the keyToBinary branch April 9, 2019 20:17

sfilipi added a commit to sfilipi/machinelearning-1 that referenced this pull request Apr 9, 2019

Key to binary samples for documentation (dotnet#3211)

087157e

* adding samples for KeyToVector and KeyToBinaryVector * typo * Adding pointers to the KeyType. * PR review comments

sfilipi mentioned this pull request Apr 9, 2019

Release cherry picks for Conversions samples and removing gplv2 files #3264

Closed

TomFinley mentioned this pull request Apr 12, 2019

Cherry-pick bug fixes and samples from master to release/1.0 #3303

Closed

ghost locked as resolved and limited conversation to collaborators Mar 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Key to binary samples for documentation #3211

Key to binary samples for documentation #3211

sfilipi commented Apr 5, 2019

sfilipi Apr 5, 2019 •

edited

Loading

codecov bot commented Apr 5, 2019 •

edited

Loading

sfilipi commented Apr 5, 2019 •

edited

Loading

wschin Apr 6, 2019 •

edited by sfilipi

Loading

sfilipi Apr 7, 2019

rogancarr commented Apr 8, 2019 •

edited by sfilipi

Loading

rogancarr Apr 8, 2019 •

edited by sfilipi

Loading

sfilipi Apr 9, 2019

rogancarr left a comment •

edited

Loading

artidoro Apr 8, 2019 •

edited by sfilipi

Loading

artidoro Apr 8, 2019 •

edited by sfilipi

Loading

artidoro Apr 8, 2019 •

edited

Loading

artidoro left a comment

sfilipi commented Apr 9, 2019

Key to binary samples for documentation #3211

Key to binary samples for documentation #3211

Conversation

sfilipi commented Apr 5, 2019

sfilipi Apr 5, 2019 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Apr 5, 2019 • edited Loading

Codecov Report

sfilipi commented Apr 5, 2019 • edited Loading

wschin Apr 6, 2019 • edited by sfilipi Loading

Choose a reason for hiding this comment

sfilipi Apr 7, 2019

Choose a reason for hiding this comment

rogancarr commented Apr 8, 2019 • edited by sfilipi Loading

rogancarr Apr 8, 2019 • edited by sfilipi Loading

Choose a reason for hiding this comment

sfilipi Apr 9, 2019

Choose a reason for hiding this comment

rogancarr left a comment • edited Loading

Choose a reason for hiding this comment

artidoro Apr 8, 2019 • edited by sfilipi Loading

Choose a reason for hiding this comment

artidoro Apr 8, 2019 • edited by sfilipi Loading

Choose a reason for hiding this comment

artidoro Apr 8, 2019 • edited Loading

Choose a reason for hiding this comment

artidoro left a comment

Choose a reason for hiding this comment

sfilipi commented Apr 9, 2019

sfilipi Apr 5, 2019 •

edited

Loading

codecov bot commented Apr 5, 2019 •

edited

Loading

sfilipi commented Apr 5, 2019 •

edited

Loading

wschin Apr 6, 2019 •

edited by sfilipi

Loading

rogancarr commented Apr 8, 2019 •

edited by sfilipi

Loading

rogancarr Apr 8, 2019 •

edited by sfilipi

Loading

rogancarr left a comment •

edited

Loading

artidoro Apr 8, 2019 •

edited by sfilipi

Loading

artidoro Apr 8, 2019 •

edited by sfilipi

Loading

artidoro Apr 8, 2019 •

edited

Loading