-
Notifications
You must be signed in to change notification settings - Fork 1.9k
In-memory & self-contained sample template. #2979
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -20,18 +20,48 @@ public static void Example() | |
var examples = GenerateRandomDataPoints(1000); | ||
|
||
// Convert the examples list to an IDataView object, which is consumable by ML.NET API. | ||
var data = mlContext.Data.LoadFromEnumerable(examples); | ||
var trainingData = mlContext.Data.LoadFromEnumerable(examples); | ||
|
||
// Define the trainer. | ||
var pipeline = mlContext.BinaryClassification.Trainers.FastTree(); | ||
var pipeline = mlContext.Regression.Trainers.FastTree(); | ||
|
||
// Train the model. | ||
var model = pipeline.Fit(data); | ||
var model = pipeline.Fit(trainingData); | ||
|
||
// Create testing examples. Use different random seed to make it different from training data. | ||
var testData = mlContext.Data.LoadFromEnumerable(GenerateRandomDataPoints(500, seed:123)); | ||
|
||
// Run the model on test data set. | ||
var transformedTestData = model.Transform(testData); | ||
|
||
// Convert IDataView object to a list. | ||
var predictions = mlContext.Data.CreateEnumerable<Prediction>(transformedTestData, reuseRowObject: false).ToList(); | ||
|
||
// Look at 5 predictions | ||
foreach (var p in predictions.Take(5)) | ||
Console.WriteLine($"Label: {p.Label:F3}, Prediction: {p.Score:F3}"); | ||
|
||
// Expected output: | ||
// Label: 0.985, Prediction: 0.938 | ||
// Label: 0.155, Prediction: 0.131 | ||
// Label: 0.515, Prediction: 0.517 | ||
// Label: 0.566, Prediction: 0.519 | ||
// Label: 0.096, Prediction: 0.089 | ||
|
||
// Evaluate the overall metrics | ||
var metrics = mlContext.Regression.Evaluate(transformedTestData); | ||
SamplesUtils.ConsoleUtils.PrintMetrics(metrics); | ||
|
||
// Expected output: | ||
// Mean Absolute Error: 0.05 | ||
// Mean Squared Error: 0.00 | ||
// Root Mean Squared Error: 0.06 | ||
// RSquared: 0.95 | ||
} | ||
|
||
private static IEnumerable<DataPoint> GenerateRandomDataPoints(int count) | ||
private static IEnumerable<DataPoint> GenerateRandomDataPoints(int count, int seed=0) | ||
{ | ||
var random = new Random(0); | ||
var random = new Random(seed); | ||
float randomFloat() => (float)random.NextDouble(); | ||
for (int i = 0; i < count; i++) | ||
{ | ||
|
@@ -45,11 +75,21 @@ private static IEnumerable<DataPoint> GenerateRandomDataPoints(int count) | |
} | ||
} | ||
|
||
// Example with label and 50 feature values. A data set is a collection of such examples. | ||
private class DataPoint | ||
{ | ||
public float Label { get; set; } | ||
[VectorType(50)] | ||
public float[] Features { get; set; } | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I assume all the samples are going to use same DataPoints (to be consistent), right? #Resolved There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess so. We now have samples, examples, instances, which are kind of less precise than data point. #Resolved There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the label type and number of features might change depending on the scenario, but the outline is the same. In reply to: 266160360 [](ancestors = 266160360) |
||
|
||
// Class used to capture predictions. | ||
private class Prediction | ||
{ | ||
// Original label. | ||
public float Label { get; set; } | ||
// Predicted score from the trainer. | ||
public float Score { get; set; } | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like extending the DataPoint class, because that is effectively what happens, columns get added. not a biggie though. #Resolved There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There's cons and pros here. I think keeping input and output separate is easier to understand for users. In reply to: 266150723 [](ancestors = 266150723) |
||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i like this, but do you think it will be quick to create them artificially for each task? Ranking and time series come to mind. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sometimes it's not easy, ranking being an example. For those, I'll keep the text-loader style.
so this template is mostly suitable for regression and binary classification.
In reply to: 266151216 [](ancestors = 266151216)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Text data is also not easy to randomly generate.
In reply to: 266157156 [](ancestors = 266157156,266151216)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe Zeeshan A meant meaningful text data but I don't think we need meaningful data to demonstrate the functionality of a module. The amount of data might be a problem to trainers, but to my knowledge, there is no trainer directly consuming strings.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sample is just a template. If in some scenarios it doesn't make sense, we can try text-loader instead.
In reply to: 266160094 [](ancestors = 266160094,266157156,266151216)