WordTokenizeingTransformer need example of how to use it.

[wschin]

I am looking for examples for using WordTokenizingTransformer. The only resources I found is its test, but after reading that, I still don't know how to print out its output using Console.WriteLine. As a C# library, we need more end-to-end examples which feed C# data structures and get the produced results as native C# objects.

[sfilipi]

@wschin what should we define as completion criteria for this bug: adding a sample for WordTokenizingTransformer?

[wschin]

@sfilipi, yes and in the future, every estimator should have some tests for native world. We need to demonstrate/test our ways to send something to the universe and get them back.

[Ivanidzo4ka]

@rogancarr @sfilipi @shmoradims Do we have list of all required samples to our API, so we can link this issue to it?

[rogancarr]

We're adding samples & docs requirements to #1209

[jwood803]

Just curious as I was messing with this, is the WordTokenizingEstimator used for the older static API? Should the context.Transforms.Text.TokenizeWords transform be used instead for the newer dynamic API?

[rogancarr]

The catalog entries are helper functions for calling the Estimator API.

[shmoradims]

Already fixed: https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.textcatalog.tokenizeintowords?view=ml-dotnet#examples