Created samples for 'FeaturizeText' API. #3120
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shmoradims
reviewed
Mar 27, 2019
| // as well as the source of randomness. | ||
| var mlContext = new MLContext(); | ||
|
|
||
| // Get a small dataset as an IEnumerable. |
There was a problem hiding this comment.
Get [](start = 15, length = 3)
Create / Define #Resolved
shmoradims
reviewed
Mar 27, 2019
| // Get a small dataset as an IEnumerable. | ||
| var samples = new List<TextData>() | ||
| { | ||
| new TextData(){ Text ="ML.NET's FeaturizeText API uses a composition of several basic transforms to convert text into numeric features." }, |
Codecov Report
@@ Coverage Diff @@
## master #3120 +/- ##
==========================================
- Coverage 72.52% 72.51% -0.01%
==========================================
Files 808 808
Lines 144665 144665
Branches 16198 16198
==========================================
- Hits 104913 104910 -3
- Misses 35342 35344 +2
- Partials 4410 4411 +1
|
Codecov Report
@@ Coverage Diff @@
## master #3120 +/- ##
=======================================
Coverage 72.52% 72.52%
=======================================
Files 808 808
Lines 144665 144665
Branches 16198 16198
=======================================
Hits 104913 104913
+ Misses 35342 35341 -1
- Partials 4410 4411 +1
|
shmoradims
reviewed
Mar 27, 2019
| // Use ML.NET's built-in stop word remover | ||
| StopWordsRemoverOptions = new StopWordsRemovingEstimator.Options() { Language = TextFeaturizingEstimator.Language.English }, | ||
| WordFeatureExtractor = new WordBagEstimator.Options() { NgramLength = 1 }, | ||
| CharFeatureExtractor = new WordBagEstimator.Options() { NgramLength = 1 }, |
There was a problem hiding this comment.
1 [](start = 86, length = 1)
is this single char tokenization? it would be just the alphabets. is it ever useful? #Resolved
There was a problem hiding this comment.
Yes, can be useful in some cases. but lets change it to 3-gram which is more useful.
In reply to: 269776047 [](ancestors = 269776047)
|
Thanks! |
zeahmed
added a commit
to zeahmed/machinelearning
that referenced
this pull request
Apr 8, 2019
ghost
locked as resolved and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to #1209.