ValueMappingEstimator example #2222

singlis · 2019-01-24T01:42:18Z

This provides an example that demonstrates different ways to use the
ValueMappingEstimator. This is part of the original change to add the
ValueMappingEstimator to the code base and references #754.

This provides an example that demonstrates different ways to use the ValueMappingEstimator. This is part of the original change to add the ValueMappingEstimator to the code base and references dotnet#754.

rogancarr · 2019-01-24T03:55:44Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+        ///     <item>12+yrs -> Cat3</item>
+        /// </list>
+        /// </summary>
+        public static void StringToStringMappingExample(MLContext ml, IDataView trainData)


I might invert these samples so that each sample is a runnable unit. Have the data loader be the helper function, rather than the example be the helper function. #Resolved

I moved these to be a runnable unit. The challenge is the associated internal classes that we use to load the data back in. So here is what I have done: I put a single example in ValueMapping.cs -- this is what the documentation will refer to. Then I added a second file called ValueMappingExtra that contains the remaining functions. Each function is now standalone that you can copy and paste, but you will need to also copy the associated class.

In reply to: 250459000 [](ancestors = 250459000)

rogancarr · 2019-01-24T03:56:31Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+                "12+yrs"
+            };
+
+            var educationValues = new List<string>()


Explain what these are, just as you explained what the keys were. #Resolved

rogancarr · 2019-01-24T03:57:59Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+                "Cat3"
+            };
+
+            var pipeline = new ValueMappingEstimator<string, string>(ml, educationKeys, educationValues, ("Education", "EducationCategory"));


Explain what this does. #Resolved

rogancarr · 2019-01-24T03:58:43Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+                "Cat2"
+            };
+
+            var pipeline = new ValueMappingEstimator<float, string>(ml, inducedKeys, inducedValues, ("Induced", "InducedCategory"));


Explain what this does. #Resolved

rogancarr · 2019-01-24T03:59:24Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+                new int[] { 42, 32 }
+            };
+
+            var pipeline = new ValueMappingEstimator<string, int>(ml, educationKeys, educationValues, ("Education", "EducationCategory"));


Explain what this does. #Resolved

rogancarr · 2019-01-24T04:00:06Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+        ///     <item>12+yrs -> 42, 32</item>
+        /// </list>
+        /// </summary>
+        public static void StringToArrayMappingExample(MLContext ml, IDataView trainData)


It might make sense to put each example in its own file so it's easier to copy and paste. #Resolved

ok - I like that better, will update...

In reply to: 250459471 [](ancestors = 250459471)

yaeldekel · 2019-01-24T17:01:26Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+            var transformedData = pipeline.Fit(trainData).Transform(trainData);
+
+            // Getting the data of the newly created column as an IEnumerable of SampleInfertDataWithFeatures.
+            var featuresColumn = ml.CreateEnumerable<SampleInfertDataWithFeatures>(transformedData, reuseRowObject: false);


featuresColumn [](start = 16, length = 14)

I find this name a bit confusing (since the data contains multiple columns). Perhaps featureRows? then it would be "foreach (featureRow in featureRows)". #Resolved

oh! good point.

In reply to: 250693972 [](ancestors = 250693972)

yaeldekel · 2019-01-24T17:03:00Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+            var pipeline = new ValueMappingEstimator<string, string>(ml, educationKeys, educationValues, ("Education", "EducationCategory"));
+
+            // The transformed data.
+            var transformedData = pipeline.Fit(trainData).Transform(trainData);


var [](start = 12, length = 3)

Write the type explicitly here to make it clearer.
Same comment for line 93. #Resolved

sfilipi · 2019-01-25T01:01:40Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+        /// level of education as keys to a respective string label which is the value.
+        /// The mapping looks like the following:
+        /// <list>
+        ///     <item>0-5yrs -> Cat1</item>


item [](start = 17, length = 4)

will need to add a tag after item, otherwise they don't render. so:

0-5yrs -> Cat1
#Resolved

- Added further comments based upon feedback

singlis · 2019-01-30T20:57:13Z

@sfilipi @yaeldekel @rogancarr Thank you for the feedback, I have posted an update, please take a look when you have a chance.

#Resolved

codecov · 2019-01-30T21:32:04Z

Codecov Report

Merging #2222 into master will increase coverage by 0.08%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #2222      +/-   ##
==========================================
+ Coverage   71.15%   71.24%   +0.08%     
==========================================
  Files         779      783       +4     
  Lines      140311   140732     +421     
  Branches    16047    16086      +39     
==========================================
+ Hits        99838   100259     +421     
+ Misses      36021    36019       -2     
- Partials     4452     4454       +2

Flag	Coverage Δ
#Debug	`71.24% <0%> (+0.08%)`	⬆️
#production	`67.6% <0%> (+0.02%)`	⬆️
#test	`85.28% <ø> (+0.19%)`	⬆️

sfilipi · 2019-01-31T18:00:09Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+        /// level of education as keys to a respective string label which is the value.
+        /// The mapping looks like the following:
+        /// <list>
+        ///     <item>0-5yrs -> Cat1</item>


0-5yrs -> Cat1 [](start = 22, length = 14)

For XML documentation, those need to be wrapped in a tag.

It doesn't matter much in your case, because this will display inside a code block. #Resolved

sfilipi · 2019-01-31T18:00:40Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+        ///     <item>0-5yrs -> Cat1</item>
+        ///     <item>6-11yrs -> Cat2</item>
+        ///     <item>12+yrs -> Cat3</item>
+        /// </list>


this should typically go in a section.

you can use simple, non-xml comments here. #Resolved

sfilipi · 2019-01-31T18:01:09Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+            var ml = new MLContext();
+
+            // Get a small dataset as an IEnumerable.
+            IEnumerable<SamplesUtils.DatasetUtils.SampleInfertData> data = SamplesUtils.DatasetUtils.GetInfertData();


IEnumerable<SamplesUtils.DatasetUtils.SampleInfertData> [](start = 12, length = 55)

var #Resolved

so the feedback was to not use var in this case because its more explicit about what we are doing. #Resolved

sfilipi · 2019-01-31T18:08:19Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+
+namespace Microsoft.ML.Samples.Dynamic
+{
+    public class ValueMappingExample


ValueMappingExample [](start = 17, length = 19)

you need to reference your samples from the extensions, otherwise they won't show up anywhere.

For how to link it, see: https://github.com/dotnet/machinelearning/blob/master/src/Microsoft.ML.StandardLearners/StandardLearnersCatalog.cs#L115

The XML of the extensions should have this on it:
///
///
///
///

#Resolved

sfilipi · 2019-01-31T18:10:28Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+            };
+
+            // Constructs the ValueMappingEstimator making the ML.net pipeline
+            var pipeline = new ValueMappingEstimator<string, string>(ml, educationKeys, educationValues, ("EducationCategory", "Education"));


new ValueMappingEstimator [](start = 27, length = 25)

i think we want to create those through the mlContext. #Resolved

Yes totally agree, I have updated these to now be called from the mlContext. However I was not able to do that for the KeyType example due to the extension methods not taking in the boolean for treatValuesAsKeyTypes. I have created an issue to track this #2346

In reply to: 252779161 [](ancestors = 252779161)

sfilipi · 2019-01-31T18:14:54Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+            // 35.0   1.0   6-11yrs     1.0         3.0       32.0     5.0  ...
+
+            // Creating a list of keys based on the Education values from the dataset
+            // These lists are created by hand for the demonstration, but the ValueMappingEstimator does take an IEnumerable.


I think the users might wonder why we are doing this, although it is somewhat explained.

Maybe try adding: if the list of keys and the list of values is known, they can be passed to the API. #Resolved

sfilipi · 2019-01-31T18:19:16Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+
+            // Features column obtained post-transformation.
+            //
+            // Age Education    EducationLabel


Age Education EducationLabel [](start = 14, length = 32)

I'd expect a column with name "EducationCategory" #Resolved

rogancarr

Hi @singlis!

Big comments:

It's not clear from these samples why anybody would actually use these transforms in practice. Can you please give examples and explanations of why these would be used?
Can you add links to these samples from the catalog entries? That will ready them for v1.
Have you seen the KeyToValue_Term.cs Dynamic Sample? This seems obsolete with your examples, so maybe remove it.

rogancarr · 2019-01-31T18:06:37Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+        {
+            // Create a new ML context, for ML.NET operations. It can be used for exception tracking and logging, 
+            // as well as the source of randomness.
+            var ml = new MLContext();


mlContext #Resolved

rogancarr · 2019-01-31T18:07:52Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+            public string EducationCategory = default;
+        }
+
+        ///<summary>


This sample is missing the "So what?" aspect. Why would I use this transform, and where is it helpful?

Can you give an example of a problem you can't solve without using this? #Resolved

rogancarr · 2019-01-31T18:08:35Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+            // 42  0-5yrs       Cat1
+            // 39  12+yrs       Cat3
+            // 34  0-5yrs       Cat1
+            // 35  6-11yrs      Cat2


Related to my comment above, what is the use of changing it from a string to a different string? Can you show what this can then be used for? #Resolved

Added a remarks section and added additional verbage to each example.

In reply to: 252778545 [](ancestors = 252778545)

rogancarr · 2019-01-31T18:09:34Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingFloatToString.cs

+
+namespace Microsoft.ML.Samples.Dynamic
+{
+    public class ValueMappingFloatToStringExample


Same comment as above — what problem does this solve? #Resolved

rogancarr · 2019-01-31T18:09:53Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToArray.cs

+
+namespace Microsoft.ML.Samples.Dynamic
+{
+    public class ValueMappingStringToArrayExample


Same comment as above — what problem does this solve? #Resolved

rogancarr · 2019-01-31T18:13:11Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingFloatToString.cs

+        {
+            // Create a new ML context, for ML.NET operations. It can be used for exception tracking and logging, 
+            // as well as the source of randomness.
+            var ml = new MLContext();


mlContext #Resolved

rogancarr · 2019-01-31T18:16:25Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToArray.cs

+            };
+
+            // Constructs the ValueMappingEstimator making the ML.net pipeline
+            var pipeline = new ValueMappingEstimator<string, int>(ml, educationKeys, educationValues, ("EducationCategory", "Education"));


Can you call this from mlContext ? #Resolved

rogancarr · 2019-01-31T18:18:21Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToKeyType.cs

+            // The KeyToValueMappingEstimator is added to provide a reverse lookup of the KeyType, converting the KeyType value back
+            // to the original value.
+            var pipeline = new ValueMappingEstimator<string, string>(ml, educationKeys, educationValues, true, ("EducationKeyType", "Education"))
+                              .Append(new KeyToValueMappingEstimator(ml, ("EducationCategory", "EducationKeyType")));


use mlContext. #Resolved

rogancarr · 2019-01-31T18:18:54Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingFloatToString.cs

+            };
+
+            // Constructs the ValueMappingEstimator making the ML.net pipeline
+            var pipeline = new ValueMappingEstimator<float, string>(ml, inducedKeys, inducedValues, ("InducedCategory", "Induced"));


Can you call this from mlContext? #Resolved

This should be updated -- there are two cases that I can not call from mlContext. I have created issues for that.

In reply to: 252782089 [](ancestors = 252782089)

rogancarr · 2019-01-31T18:19:08Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+            };
+
+            // Constructs the ValueMappingEstimator making the ML.net pipeline
+            var pipeline = new ValueMappingEstimator<string, string>(ml, educationKeys, educationValues, ("EducationCategory", "Education"));


Can you call this from mlContext? #Resolved

sfilipi · 2019-01-31T18:22:19Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToArray.cs

+            };
+
+            // Constructs the ValueMappingEstimator making the ML.net pipeline
+            var pipeline = new ValueMappingEstimator<string, int>(ml, educationKeys, educationValues, ("EducationCategory", "Education"));


r<string, int> [](start = 51, length = 14)

can we merge this sample with the other one? maybe have a pipeline with the two estimators?

can we merge all this samples into one, with a pipeline with more than one estimator? #ByDesign

- Added remarks section to ValueMapping - Other updates based upon feedback

rogancarr

Approved with comments.

rogancarr · 2019-01-31T23:20:27Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToArray.cs

+            IDataView trainData = mlContext.Data.ReadFromEnumerable(data);
+
+            // If the list of keys and values are known, they can be passed to the API. The ValueMappingEstimator can also get the mapping through an IDataView
+


nit: Empty line #Resolved

rogancarr · 2019-01-31T23:21:22Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToArray.cs

+        }
+
+        /// This example demonstrates the use of the ValueMappingEstimator by mapping string-to-array values which allows for mapping string data
+        /// to numeric arrays that can then be used as a feature set for a trainer. In this example, we are mapping the education data to


As a feature set for a trainer

Does this work? The vectors are of different sizes. #Resolved

No probably not to pass into a trainer. I went ahead and made them the same size.

In reply to: 252881242 [](ancestors = 252881242)

rogancarr · 2019-01-31T23:21:44Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToKeyType.cs

+        /// instead of the actual value provides a unique integer representation of the value. When the treatValueAsKeyTypes is true, 
+        /// the ValueMappingEstimator will generate a KeyType for each unique value.
+        /// 
+        /// In this example, the education data is mapped to a grouping of 'Undergraudate' and 'Postgraduate'. Because KeyTypes are used, the


Undergraduate #Resolved

sfilipi · 2019-01-31T23:57:39Z

src/Microsoft.ML.Data/Transforms/ValueMapping.cs

-    /// The ValueMappingEstimator is a 1-1 mapping from a key to value. This particular class load the mappings from an <see cref="IDataView"/>.
-    /// This gives user the flexibility to load the mapping from file instead of using IEnumerable in <see cref="ValueMappingEstimator{TKey, TValue}"/>
-    /// </summary>
+    /// <include file='doc.xml' path='doc/members/member[@name="ValueMappingEstimator"]/*' />
    public class ValueMappingEstimator : TrivialEstimator<ValueMappingTransformer>


ValueMappingEstimator [](start = 17, length = 21)

i'd put it in the transformer too. #Resolved

put the comments for the estimator on the transformer? Or create a new entry for the transformer?

In reply to: 252889880 [](ancestors = 252889880)

Added to transformer.

In reply to: 252937283 [](ancestors = 252937283,252889880)

sfilipi · 2019-01-31T23:58:10Z

src/Microsoft.ML.Data/Transforms/doc.xml

+        specific value. The ValueMappingEstimator supports keys and values of different <see cref="System.Type"/> to support different data types.
+        Examples for using a ValueMappingEstimator are:
+        <list>
+          <li>


li [](start = 11, length = 2)

item #Resolved

sfilipi · 2019-01-31T23:59:53Z

    /// Maps specified keys to specified values

the text in the doc.xml seems more complete. maybe use that, or #Resolved

Refers to: src/Microsoft.ML.Data/Transforms/ConversionsExtensionsCatalog.cs:139 in b3d1df2. [](commit_id = b3d1df2, deletion_comment = False)

sfilipi · 2019-02-01T00:00:42Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMapping.cs

+        }
+
+        /// This example demonstrates the use of the ValueMappingEstimator by mapping string-to-string values. This is useful
+        /// to map strings to a grouping. In this example, the Education data maps to the groups Undergraduate and Postgraduate:


grouping [](start = 32, length = 8)

shall we use the term 'key' #ByDesign

are you wanting it to say "map keys to a grouping"?

In reply to: 252890528 [](ancestors = 252890528)

sfilipi · 2019-02-01T00:01:55Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToKeyType.cs

+            // Generate the ValueMappingEstimator that will output KeyTypes even though our values are strings.
+            // The KeyToValueMappingEstimator is added to provide a reverse lookup of the KeyType, converting the KeyType value back
+            // to the original value.
+            var pipeline = new ValueMappingEstimator<string, string>(mlContext, educationKeys, educationValues, true, ("EducationKeyType", "Education"))


new ValueMappingEstimator< [](start = 27, length = 26)

use mlContext #ByDesign

I need to add an api to the extensions catalog to allow for the treatValuesAsKeyTypes. I dont want to add that in this PR as I am trying to scope this to documentation changes. I have a github issue tracking it and update this sample once its fixed

In reply to: 252890986 [](ancestors = 252890986)

sfilipi · 2019-02-01T00:03:02Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToArray.cs

+            };
+
+            // Constructs the ValueMappingEstimator making the ML.net pipeline
+            var pipeline = new ValueMappingEstimator<string, int>(mlContext, educationKeys, educationValues, ("EducationFeature", "Education"));


new ValueMappingEstimator [](start = 27, length = 25)

mlContext #ByDesign

same deal here -- need to add an api to support vector types. I have a github issue to fix.

In reply to: 252891208 [](ancestors = 252891208)

sfilipi · 2019-02-01T00:03:14Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToArray.cs

+            IDataView transformedData = pipeline.Fit(trainData).Transform(trainData);
+
+            // Getting the resulting data as an IEnumerable of SampleInfertDataWithIntArray. This will contain the newly created column EducationCategory
+            IEnumerable<SampleInfertDataWithIntArray> featuresColumn = mlContext.CreateEnumerable<SampleInfertDataWithIntArray>(transformedData, reuseRowObject: false);


IEnumerable<SampleInfertDataWithIntArray [](start = 12, length = 40)

nit;: var #Resolved

Yael's feedback was to keep this explicit for demo purposes. What would you prefer?

In reply to: 252891247 [](ancestors = 252891247)

opinions, i guess :)

In reply to: 252938037 [](ancestors = 252938037,252891247)

:) ok I will keep as-is.

In reply to: 252938544 [](ancestors = 252938544,252938037,252891247)

sfilipi · 2019-02-01T00:04:15Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingStringToArray.cs

+
+namespace Microsoft.ML.Samples.Dynamic
+{
+    public class ValueMappingStringToArrayExample


ValueMappingStringToArrayExample [](start = 17, length = 32)

this is not referenced anywhere. #Resolved

RIght - this is just another example that is available to show how this can be used. There is another example that is not used either. Does they all need to be referenced?

In reply to: 252891445 [](ancestors = 252891445)

I think we should remove them. The purpose of this project is to populate the API reference website.

the machinelearning-samples repo is the one where we can put more samples, if needed.

If we keep them here, unreferenced, it will be hard to see what is used for what, overtime.

In reply to: 252938208 [](ancestors = 252938208,252891445)

Is it possible to add multiple links? I initially had these in one file but moved out into separate files to not have to update line number reference.

In reply to: 252938844 [](ancestors = 252938844,252938208,252891445)

I added multiple links so they are all referenced now.

In reply to: 252939988 [](ancestors = 252939988,252938844,252938208,252891445)

sfilipi · 2019-02-01T00:04:41Z

docs/samples/Microsoft.ML.Samples/Dynamic/ValueMappingFloatToString.cs

+
+namespace Microsoft.ML.Samples.Dynamic
+{
+    public class ValueMappingFloatToStringExample


ValueMappingFloatToStringExample [](start = 17, length = 32)

this is not referenced from anything. #Resolved

sfilipi

sfilipi

singlis · 2019-02-01T05:55:39Z

    /// Maps specified keys to specified values

so I added a see cref in the summary tag but not sure how that will render.

In reply to: 459553687 [](ancestors = 459553687)

Refers to: src/Microsoft.ML.Data/Transforms/ConversionsExtensionsCatalog.cs:139 in b3d1df2. [](commit_id = b3d1df2, deletion_comment = False)

ValueMappingEstimator example

bfdbc4f

This provides an example that demonstrates different ways to use the ValueMappingEstimator. This is part of the original change to add the ValueMappingEstimator to the code base and references dotnet#754.

singlis self-assigned this Jan 24, 2019

singlis requested review from codemzs, Ivanidzo4ka, wschin, yaeldekel, sfilipi, rogancarr and zeahmed January 24, 2019 01:42

rogancarr reviewed Jan 24, 2019

View reviewed changes

codemzs closed this Jan 24, 2019

codemzs reopened this Jan 24, 2019

yaeldekel reviewed Jan 24, 2019

View reviewed changes

sfilipi reviewed Jan 25, 2019

View reviewed changes

singlis added 2 commits January 30, 2019 11:43

Merge remote-tracking branch 'origin/master' into singlis/value-example

9ad04ff

- Moved all tests into individual files

9159ae5

- Added further comments based upon feedback

sfilipi reviewed Jan 31, 2019

View reviewed changes

rogancarr reviewed Jan 31, 2019

View reviewed changes

sfilipi reviewed Jan 31, 2019

View reviewed changes

sfilipi mentioned this pull request Jan 31, 2019

API reference - Samples for Transforms #1209

Closed

- Added CDATA links to the examples for ValueMapping and KeyToValue

beb48eb

- Added remarks section to ValueMapping - Other updates based upon feedback

...updating from feedback

b3d1df2

rogancarr approved these changes Jan 31, 2019

View reviewed changes

sfilipi reviewed Jan 31, 2019

View reviewed changes

sfilipi reviewed Feb 1, 2019

View reviewed changes

sfilipi approved these changes Feb 1, 2019

View reviewed changes

singlis added 2 commits January 31, 2019 21:57

..updating..

a50a29f

...updating...

8542762

singlis merged commit 578c188 into dotnet:master Feb 1, 2019

singlis deleted the singlis/value-example branch March 7, 2019 00:54

ghost locked as resolved and limited conversation to collaborators Mar 25, 2022

		IDataView trainData = mlContext.Data.ReadFromEnumerable(data);

		// If the list of keys and values are known, they can be passed to the API. The ValueMappingEstimator can also get the mapping through an IDataView

ValueMappingEstimator example #2222

ValueMappingEstimator example #2222

Conversation

singlis commented Jan 24, 2019

rogancarr Jan 24, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Jan 24, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 24, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 24, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 24, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 24, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yaeldekel Jan 24, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yaeldekel Jan 24, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

sfilipi Jan 25, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

singlis commented Jan 30, 2019 • edited Loading

codecov bot commented Jan 30, 2019 • edited Loading

Codecov Report

sfilipi Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

sfilipi Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

sfilipi Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

singlis Jan 31, 2019 • edited Loading

Choose a reason for hiding this comment

sfilipi Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

sfilipi Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfilipi Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

sfilipi Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr left a comment

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

sfilipi Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr left a comment

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rogancarr Jan 31, 2019 • edited by singlis Loading

Choose a reason for hiding this comment

rogancarr Jan 24, 2019 •

edited by singlis

Loading

rogancarr Jan 24, 2019 •

edited by singlis

Loading

rogancarr Jan 24, 2019 •

edited by singlis

Loading

rogancarr Jan 24, 2019 •

edited by singlis

Loading

rogancarr Jan 24, 2019 •

edited by singlis

Loading

rogancarr Jan 24, 2019 •

edited by singlis

Loading

yaeldekel Jan 24, 2019 •

edited by singlis

Loading

yaeldekel Jan 24, 2019 •

edited by singlis

Loading

sfilipi Jan 25, 2019 •

edited by singlis

Loading

singlis commented Jan 30, 2019 •

edited

Loading

codecov bot commented Jan 30, 2019 •

edited

Loading

sfilipi Jan 31, 2019 •

edited by singlis

Loading

sfilipi Jan 31, 2019 •

edited by singlis

Loading

sfilipi Jan 31, 2019 •

edited by singlis

Loading

singlis Jan 31, 2019 •

edited

Loading

sfilipi Jan 31, 2019 •

edited by singlis

Loading

sfilipi Jan 31, 2019 •

edited by singlis

Loading

sfilipi Jan 31, 2019 •

edited by singlis

Loading

sfilipi Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

sfilipi Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

rogancarr Jan 31, 2019 •

edited by singlis

Loading

sfilipi Jan 31, 2019 •

edited by singlis

Loading

sfilipi Jan 31, 2019 •

edited by singlis

Loading

sfilipi commented Jan 31, 2019 •

edited by singlis

Loading

sfilipi Feb 1, 2019 •

edited by singlis

Loading

sfilipi Feb 1, 2019 •

edited by singlis

Loading

sfilipi Feb 1, 2019 •

edited by singlis

Loading

sfilipi Feb 1, 2019 •

edited by singlis

Loading

sfilipi Feb 1, 2019 •

edited by singlis

Loading

sfilipi Feb 1, 2019 •

edited by singlis

Loading