Skip to content

In-memory & self-contained sample template. #2979

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 18, 2019
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,48 @@ public static void Example()
var examples = GenerateRandomDataPoints(1000);

// Convert the examples list to an IDataView object, which is consumable by ML.NET API.
var data = mlContext.Data.LoadFromEnumerable(examples);
var trainingData = mlContext.Data.LoadFromEnumerable(examples);

// Define the trainer.
var pipeline = mlContext.BinaryClassification.Trainers.FastTree();
var pipeline = mlContext.Regression.Trainers.FastTree();

// Train the model.
var model = pipeline.Fit(data);
var model = pipeline.Fit(trainingData);

// Create testing examples. Use different random seed to make it different from training data.
var testData = mlContext.Data.LoadFromEnumerable(GenerateRandomDataPoints(500, seed:123));

// Run the model on test data set.
var transformedTestData = model.Transform(testData);

// Convert IDataView object to a list.
var predictions = mlContext.Data.CreateEnumerable<Prediction>(transformedTestData, reuseRowObject: false).ToList();

// Look at 5 predictions
foreach (var p in predictions.Take(5))
Console.WriteLine($"Label: {p.Label:F3}, Prediction: {p.Score:F3}");

// Expected output:
// Label: 0.985, Prediction: 0.938
// Label: 0.155, Prediction: 0.131
// Label: 0.515, Prediction: 0.517
// Label: 0.566, Prediction: 0.519
// Label: 0.096, Prediction: 0.089

// Evaluate the overall metrics
var metrics = mlContext.Regression.Evaluate(transformedTestData);
SamplesUtils.ConsoleUtils.PrintMetrics(metrics);

// Expected output:
// Mean Absolute Error: 0.05
// Mean Squared Error: 0.00
// Root Mean Squared Error: 0.06
// RSquared: 0.95
}

private static IEnumerable<DataPoint> GenerateRandomDataPoints(int count)
private static IEnumerable<DataPoint> GenerateRandomDataPoints(int count, int seed=0)
Copy link
Member

@sfilipi sfilipi Mar 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

private static IEnumerable GenerateRandomDataPoints(int count, int seed=0) [](start = 7, length = 86)

i like this, but do you think it will be quick to create them artificially for each task? Ranking and time series come to mind. #Resolved

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sometimes it's not easy, ranking being an example. For those, I'll keep the text-loader style.

so this template is mostly suitable for regression and binary classification.


In reply to: 266151216 [](ancestors = 266151216)

Copy link
Contributor

@zeahmed zeahmed Mar 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Text data is also not easy to randomly generate.


In reply to: 266157156 [](ancestors = 266157156,266151216)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe Zeeshan A meant meaningful text data but I don't think we need meaningful data to demonstrate the functionality of a module. The amount of data might be a problem to trainers, but to my knowledge, there is no trainer directly consuming strings.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sample is just a template. If in some scenarios it doesn't make sense, we can try text-loader instead.


In reply to: 266160094 [](ancestors = 266160094,266157156,266151216)

{
var random = new Random(0);
var random = new Random(seed);
float randomFloat() => (float)random.NextDouble();
for (int i = 0; i < count; i++)
{
Expand All @@ -45,11 +75,21 @@ private static IEnumerable<DataPoint> GenerateRandomDataPoints(int count)
}
}

// Example with label and 50 feature values. A data set is a collection of such examples.
private class DataPoint
{
public float Label { get; set; }
[VectorType(50)]
public float[] Features { get; set; }
}
Copy link
Contributor

@zeahmed zeahmed Mar 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume all the samples are going to use same DataPoints (to be consistent), right? #Resolved

Copy link
Member

@wschin wschin Mar 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess so. We now have samples, examples, instances, which are kind of less precise than data point. #Resolved

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the label type and number of features might change depending on the scenario, but the outline is the same.


In reply to: 266160360 [](ancestors = 266160360)


// Class used to capture predictions.
private class Prediction
{
// Original label.
public float Label { get; set; }
// Predicted score from the trainer.
public float Score { get; set; }
}
Copy link
Member

@sfilipi sfilipi Mar 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like extending the DataPoint class, because that is effectively what happens, columns get added. not a biggie though. #Resolved

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's cons and pros here. I think keeping input and output separate is easier to understand for users.


In reply to: 266150723 [](ancestors = 266150723)

}
}