Stopwords remover xml documentation #3413

Ivanidzo4ka · 2019-04-18T21:43:51Z

towards #3204

codecov · 2019-04-18T22:24:12Z

Codecov Report

Merging #3413 into master will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3413      +/-   ##
==========================================
+ Coverage   72.69%   72.73%   +0.03%     
==========================================
  Files         807      807              
  Lines      145171   145206      +35     
  Branches    16225    16230       +5     
==========================================
+ Hits       105536   105612      +76     
+ Misses      35220    35176      -44     
- Partials     4415     4418       +3

Flag	Coverage Δ
#Debug	`72.73% <ø> (+0.03%)`	⬆️
#production	`68.27% <ø> (+0.04%)`	⬆️
#test	`88.98% <ø> (ø)`	⬆️

Impacted Files	Coverage Δ
src/Microsoft.ML.Transforms/Text/TextCatalog.cs	`41.66% <ø> (ø)`	⬆️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs	`86.1% <ø> (ø)`	⬆️
src/Microsoft.ML.Data/Transforms/KeyToValue.cs	`79.16% <0%> (-0.65%)`	⬇️
...ML.Transforms/MutualInformationFeatureSelection.cs	`78.58% <0%> (-0.33%)`	⬇️
...soft.ML.Tests/Transformers/CategoricalHashTests.cs	`100% <0%> (ø)`	⬆️
src/Microsoft.ML.Transforms/NormalizerCatalog.cs	`84.78% <0%> (ø)`	⬆️
...osoft.ML.Tests/Transformers/TextFeaturizerTests.cs	`99.58% <0%> (ø)`	⬆️
...soft.ML.Transforms/Text/NgramHashingTransformer.cs	`88.79% <0%> (+0.01%)`	⬆️
...soft.ML.TestFramework/DataPipe/TestDataPipeBase.cs	`73.73% <0%> (+0.02%)`	⬆️
... and 5 more

Ivanidzo4ka · 2019-04-18T22:41:03Z

src/Microsoft.ML.Transforms/Text/StopWordsRemovingTransformer.cs

+    /// | Output column data type | Vector of unknown size of [Text]<xref:System.ReadOnlyMemory{System.Char}> |
+    ///
+    /// The resulting <xref:Microsoft.ML.Transforms.Text.StopWordsRemovingTransformer/> creates a new column, named as specified in the output column name parameters, and
+    /// fills it with vector of strings similar to vector of strings in input column but removing all, predefined for certain language, strings from it.


strings [](start = 32, length = 7)

replace strings with texts? #Resolved

shmoradims · 2019-04-19T00:02:28Z

src/Microsoft.ML.Transforms/Text/StopWordsRemovingTransformer.cs

+    /// |  |  |
+    /// | -- | -- |
+    /// | Does this estimator need to look at the data to train its parameters? | No |
+    /// | Input column data type | Vector of [Text]<xref:System.ReadOnlyMemory{System.Char}> |


[Text]xref:System.ReadOnlyMemory{System.Char} [](start = 45, length = 47)

nice. let's check how this renders when the preview site is refreshed tomorrow

shmoradims · 2019-04-19T00:03:00Z

src/Microsoft.ML.Transforms/Text/StopWordsRemovingTransformer.cs

+    /// |  |  |
+    /// | -- | -- |
+    /// | Does this estimator need to look at the data to train its parameters? | No |
+    /// | Input column data type | Vector of [Text]<xref:System.ReadOnlyMemory{System.Char}> |


Vector [](start = 35, length = 6)

should this be known-sized? #ByDesign

No, it can be either known-sized or unknown size.
output would always be unknown size

In reply to: 276871092 [](ancestors = 276871092)

shmoradims · 2019-04-19T00:06:09Z

src/Microsoft.ML.Transforms/Text/TextCatalog.cs

-        /// Removes stop words from incoming token streams in <paramref name="inputColumnName"/>
-        /// and outputs the token streams without stopwords as <paramref name="outputColumnName"/>.
+        /// Create a <see cref="CustomStopWordsRemovingEstimator"/>, which copies the data from the column specified in <paramref name="inputColumnName"/>
+        /// to a new column: <paramref name="outputColumnName"/> and removes predifined set of text specific for <paramref name="language"/> from it.


specific for [](start = 100, length = 40)

for later: we should that these language-specific lists to docs.

shmoradims · 2019-04-19T00:06:51Z

src/Microsoft.ML.Transforms/Text/StopWordsRemovingTransformer.cs

+    /// | -- | -- |
+    /// | Does this estimator need to look at the data to train its parameters? | No |
+    /// | Input column data type | Vector of [Text]<xref:System.ReadOnlyMemory{System.Char}> |
+    /// | Output column data type | Vector of unknown size of [Text]<xref:System.ReadOnlyMemory{System.Char}> |


Vector of unknown size of [](start = 36, length = 25)

how about "Unknown-sized vector of"? #Resolved

Unspecified? #ByDesign

shmoradims

natke · 2019-04-19T01:24:08Z

src/Microsoft.ML.Transforms/Text/StopWordsRemovingTransformer.cs

+    /// | -- | -- |
+    /// | Does this estimator need to look at the data to train its parameters? | No |
+    /// | Input column data type | Vector of [Text]<xref:System.ReadOnlyMemory{System.Char}> |
+    /// | Output column data type | Unknown-sized vector of [Text]<xref:System.ReadOnlyMemory{System.Char}> |


Unspecified? #Resolved

natke · 2019-04-19T01:25:04Z

src/Microsoft.ML.Transforms/Text/StopWordsRemovingTransformer.cs

+    /// | -- | -- |
+    /// | Does this estimator need to look at the data to train its parameters? | No |
+    /// | Input column data type | Vector of [Text]<xref:System.ReadOnlyMemory{System.Char}> |
+    /// | Output column data type | Vector of unknown size of [Text]<xref:System.ReadOnlyMemory{System.Char}> |


Unspecified? #ByDesign

natke · 2019-04-19T01:26:11Z

src/Microsoft.ML.Transforms/Text/StopWordsRemovingTransformer.cs

+    /// | Output column data type | Vector of unknown size of [Text]<xref:System.ReadOnlyMemory{System.Char}> |
+    ///
+    /// The resulting <xref:Microsoft.ML.Transforms.Text.CustomStopWordsRemovingTransformer/> creates a new column, named as specified in the output column name parameters, and
+    /// fills it with vector of texts similar to vector of texts in input column but removing all specified texts from it.


specified texts -> common words? #Resolved

vector of texts -> vector of words? #Resolved

they not common, they provided by user. does that phrasing looks better?

In reply to: 276881001 [](ancestors = 276881001)

natke · 2019-04-19T20:42:24Z

src/Microsoft.ML.Transforms/Text/StopWordsRemovingTransformer.cs

+    /// | Output column data type | Unknown-sized vector of [Text](<xref:Microsoft.ML.Data.TextDataViewType>) |
+    ///
+    /// The resulting <xref:Microsoft.ML.Transforms.Text.CustomStopWordsRemovingTransformer/> creates a new column, named as specified in the output column name parameters, and
+    /// fills it with vector of texts similar to vector of words in input column but removing all provided by user words from it.


Suggest: The resulting xref:Microsoft.ML.Transforms.Text.CustomStopWordsRemovingTransformer/ creates a new column, named as specified by the output column name parameter, and
fills it with a vector of words containing all of the words in the input column except those given by the stopwords parameter. #Resolved

natke · 2019-04-19T20:45:27Z

src/Microsoft.ML.Transforms/Text/StopWordsRemovingTransformer.cs

+    /// The resulting <xref:Microsoft.ML.Transforms.Text.StopWordsRemovingTransformer/> creates a new column, named as specified in the output column name parameters, and
+    /// fills it with vector of words similar to vector of words in input column but removing all, predefined for certain language, texts from it.
+    /// All text comparison made by casting predefined text and text from input column to lower case using casing rules of invariant culture.
+    /// See the See Also section for links to examples of the usage.


Suggest: The resulting xref:Microsoft.ML.Transforms.Text.StopWordsRemovingTransformer/ creates a new column, named as specified in the output column name parameter, and fills it with a vector of words containing all of the words in the input column **except the predefined list of stopwords for the specified language. #Resolved

Looks good! Small type: extra ** in there

natke · 2019-04-20T00:13:54Z

src/Microsoft.ML.Transforms/Text/StopWordsRemovingTransformer.cs

+    /// The resulting <xref:Microsoft.ML.Transforms.Text.StopWordsRemovingTransformer/> creates a new column, named as specified in the output column name parameters, and
+    /// fills it with vector of words similar to vector of words in input column but removing all, predefined for certain language, texts from it.
+    /// All text comparison made by casting predefined text and text from input column to lower case using casing rules of invariant culture.
+    /// See the See Also section for links to examples of the usage.


Looks good! Small type: extra ** in there

Ivan Matantsev added 3 commits April 18, 2019 12:57

Update documentation for stopwords

ea847a9

Merge branch 'master' into Ivanidze/StopWordsRemoverXmlDocumentation

eb72063

string to text

7e1016c

Ivanidzo4ka added the documentation Related to documentation of ML.NET label Apr 18, 2019

Ivanidzo4ka requested review from natke, artidoro, shmoradims and sfilipi April 18, 2019 21:43

Ivanidzo4ka commented Apr 18, 2019

View reviewed changes

sfilipi mentioned this pull request Apr 18, 2019

API reference - XML documentation template for transforms #3204

Closed

shmoradims reviewed Apr 19, 2019

View reviewed changes

shmoradims approved these changes Apr 19, 2019

View reviewed changes

address comments

818ec4a

natke suggested changes Apr 19, 2019

View reviewed changes

Ivan Matantsev added 2 commits April 19, 2019 10:01

address comments

a8049d9

ReadOnly to TextDataViewType

f3f9003

Ivanidzo4ka requested a review from natke April 19, 2019 19:52

natke suggested changes Apr 19, 2019

View reviewed changes

address comments

5c5fc26

Ivanidzo4ka requested a review from natke April 19, 2019 21:15

small update for types

abff8d4

natke approved these changes Apr 20, 2019

View reviewed changes

shauheen merged commit f41e7f4 into dotnet:master Apr 20, 2019

ghost locked as resolved and limited conversation to collaborators Mar 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stopwords remover xml documentation #3413

Stopwords remover xml documentation #3413

Ivanidzo4ka commented Apr 18, 2019

codecov bot commented Apr 18, 2019 •

edited

Loading

Ivanidzo4ka Apr 18, 2019 •

edited

Loading

shmoradims Apr 19, 2019

shmoradims Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

Ivanidzo4ka Apr 19, 2019

shmoradims Apr 19, 2019

shmoradims Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

shmoradims left a comment

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

Ivanidzo4ka Apr 19, 2019

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 20, 2019

natke Apr 20, 2019

Stopwords remover xml documentation #3413

Stopwords remover xml documentation #3413

Conversation

Ivanidzo4ka commented Apr 18, 2019

codecov bot commented Apr 18, 2019 • edited Loading

Codecov Report

Ivanidzo4ka Apr 18, 2019 • edited Loading

Choose a reason for hiding this comment

shmoradims Apr 19, 2019

Choose a reason for hiding this comment

shmoradims Apr 19, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

Ivanidzo4ka Apr 19, 2019

Choose a reason for hiding this comment

shmoradims Apr 19, 2019

Choose a reason for hiding this comment

shmoradims Apr 19, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

natke Apr 19, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

shmoradims left a comment

Choose a reason for hiding this comment

natke Apr 19, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

natke Apr 19, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

natke Apr 19, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

natke Apr 19, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

Ivanidzo4ka Apr 19, 2019

Choose a reason for hiding this comment

natke Apr 19, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

natke Apr 19, 2019 • edited by Ivanidzo4ka Loading

Choose a reason for hiding this comment

natke Apr 20, 2019

Choose a reason for hiding this comment

natke Apr 20, 2019

Choose a reason for hiding this comment

codecov bot commented Apr 18, 2019 •

edited

Loading

Ivanidzo4ka Apr 18, 2019 •

edited

Loading

shmoradims Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

shmoradims Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading

natke Apr 19, 2019 •

edited by Ivanidzo4ka

Loading