Skip to content

Update xml documentation for Image estimators #3376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Apr 21, 2019
Merged

Update xml documentation for Image estimators #3376

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
bb3ed19
Update xml documentation for Image estimators
Apr 17, 2019
e2189f3
Remove doc.xml
Apr 17, 2019
5157a9d
update markdown
Apr 17, 2019
74dda82
Merge branch 'master' into Ivanidze/ImagesXmlDocumentationPassOne
Apr 18, 2019
1ed6278
update to different types
Apr 18, 2019
6d4e789
adress comments
Apr 18, 2019
911e373
address senja comments
Apr 19, 2019
7d1720b
ReadOnly.. to TextDataViewType
Apr 19, 2019
c060327
address comments
Apr 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 46 additions & 41 deletions src/Microsoft.ML.ImageAnalytics/ExtensionsCatalog.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,15 @@ namespace Microsoft.ML
{
public static class ImageEstimatorsCatalog
{
/// <include file='doc.xml' path='doc/members/member[@name="ImageGrayscalingEstimator"]/*' />
///<summary>
/// Create a <see cref="ImageGrayscalingEstimator"/>, which converts images in the column specified in <see cref="InputOutputColumnPair.InputColumnName"/>
/// to grayscale images in a new column: <see cref="InputOutputColumnPair.OutputColumnName" />.
///</summary>
/// <param name="catalog">The transform's catalog.</param>
/// <param name="outputColumnName">Name of the column resulting from the transformation of <paramref name="inputColumnName"/>.</param>
/// <param name="inputColumnName">Name of the column to transform. If set to <see langword="null"/>, the value of the <paramref name="outputColumnName"/> will be used as source.</param>
/// <param name="outputColumnName">Name of the column resulting from the transformation of <paramref name="inputColumnName"/>.
/// This column's data type will be the same as that of the input column.</param>
/// <param name="inputColumnName">Name of the column to convert images to grayscale from.
/// This estimator operates only on <see cref="System.Drawing.Bitmap"/>.</param>
/// <example>
/// <format type="text/markdown">
/// <![CDATA[
Expand All @@ -23,9 +28,12 @@ public static class ImageEstimatorsCatalog
public static ImageGrayscalingEstimator ConvertToGrayscale(this TransformsCatalog catalog, string outputColumnName, string inputColumnName = null)
=> new ImageGrayscalingEstimator(CatalogUtils.GetEnvironment(catalog), new[] { (outputColumnName, inputColumnName ?? outputColumnName) });

/// <include file='doc.xml' path='doc/members/member[@name="ImageGrayscalingEstimator"]/*' />
///<summary>
/// Create a <see cref="ImageGrayscalingEstimator"/>, which converts images in the column specified in <see cref="InputOutputColumnPair.InputColumnName"/>
Copy link
Contributor

@natke natke Apr 18, 2019 •
edited by Ivanidzo4ka
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we distinguish between the two methods? i.e. one uses the column names directly, and one supplies a column pair #ByDesign

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second method is internal, so main reason why I'm even adding anything here is because I want to get rid of doc.xml.

In reply to: 276802631 [](ancestors = 276802631)

/// to grayscale images in a new column: <see cref="InputOutputColumnPair.OutputColumnName" />.
///</summary>
/// <param name="catalog">The transform's catalog.</param>
/// <param name="columns">Specifies the names of the input columns for the transformation, and their respective output column names.</param>
/// <param name="columns">The pairs of input and output columns. This estimator operates only on <see cref="System.Drawing.Bitmap"/>.</param>
/// <example>
/// <format type="text/markdown">
/// <![CDATA[
Expand All @@ -41,20 +49,15 @@ internal static ImageGrayscalingEstimator ConvertToGrayscale(this TransformsCata
}

/// <summary>
/// Loads the images from the <see cref="ImageLoadingTransformer.ImageFolder" /> into memory.
/// Create a <see cref="ImageLoadingEstimator"/>, which loads the data from the column specified in <paramref name="inputColumnName"/>
/// as an image to a new column: <paramref name="outputColumnName"/>.
/// </summary>
/// <remarks>
/// The image get loaded in memory as a <see cref="System.Drawing.Bitmap" /> type.
/// Loading is the first step of almost every pipeline that does image processing, and further analysis on images.
/// The images to load need to be in the formats supported by <see cref = "System.Drawing.Bitmap" />.
/// For end-to-end image processing pipelines, and scenarios in your applications, see the
/// <a href="https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/getting-started"> examples in the machinelearning-samples github repository.</a>
/// <seealso cref = "ImageEstimatorsCatalog" />
/// </remarks>
/// <param name="catalog">The transform's catalog.</param>
/// <param name="outputColumnName">Name of the column resulting from the transformation of <paramref name="inputColumnName"/>.</param>
/// <param name="imageFolder">The images folder.</param>
/// <param name="inputColumnName">Name of the column to transform. If set to <see langword="null"/>, the value of the <paramref name="outputColumnName"/> will be used as source.</param>
/// <param name="outputColumnName">Name of the column resulting from the transformation of <paramref name="inputColumnName"/>.
/// This column's data type will be <see cref="System.Drawing.Bitmap"/>.</param>
/// <param name="inputColumnName">Name of the column with paths to the images to load.
/// This estimator operates over text data.</param>
Copy link
Member

@sfilipi sfilipi Apr 19, 2019 •
edited by Ivanidzo4ka
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

over text data [](start = 36, length = 14)

over text data indicating the path of the images to load. #Resolved

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rather put it in sentence above.

In reply to: 276905816 [](ancestors = 276905816)

/// <param name="imageFolder">Folder where to look for images.</param>
/// <example>
/// <format type="text/markdown">
/// <![CDATA[
Expand All @@ -73,10 +76,9 @@ public static ImageLoadingEstimator LoadImages(this TransformsCatalog catalog, s
/// The images to load need to be in the formats supported by <see cref = "System.Drawing.Bitmap" />.
/// For end-to-end image processing pipelines, and scenarios in your applications, see the
/// <a href="https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/getting-started"> examples in the machinelearning-samples github repository.</a>
/// <seealso cref = "ImageEstimatorsCatalog" />
/// </remarks>
/// <param name="catalog">The transform's catalog.</param>
/// <param name="imageFolder">The images folder.</param>
/// <param name="imageFolder">Folder where to look for images.</param>
/// <param name="columns">Specifies the names of the input columns for the transformation, and their respective output column names.</param>
/// <example>
/// <format type="text/markdown">
Expand All @@ -92,10 +94,15 @@ internal static ImageLoadingEstimator LoadImages(this TransformsCatalog catalog,
return new ImageLoadingEstimator(env, imageFolder, InputOutputColumnPair.ConvertToValueTuples(columns));
}

/// <include file='doc.xml' path='doc/members/member[@name="ImagePixelExtractingEstimator"]/*' />
/// <summary>
/// Create a <see cref="ImagePixelExtractingEstimator"/>, which extracts pixels values from the data specified in column: <paramref name="inputColumnName"/>
/// to a new column: <paramref name="outputColumnName"/>.
/// </summary>
/// <param name="catalog">The transform's catalog.</param>
/// <param name="outputColumnName">Name of the column resulting from the transformation of <paramref name="inputColumnName"/>.</param>
/// <param name="inputColumnName">Name of column to transform. If set to <see langword="null"/>, the value of the <paramref name="outputColumnName"/> will be used as source.</param>
/// <param name="outputColumnName">Name of the column resulting from the transformation of <paramref name="inputColumnName"/>.
/// This column's data type will be a known-sized vector of <see cref="System.Single"/> or <see cref="System.Byte"/> depending on <paramref name="outputAsFloatArray"/>.</param>
/// <param name="inputColumnName">Name of the column with images.
/// This estimator operates over <see cref="System.Drawing.Bitmap"/>.</param>
/// <param name="colorsToExtract">The colors to extract from the image.</param>
/// <param name="orderOfExtraction">The order in which to extract colors from pixel.</param>
/// <param name="interleavePixelColors">Whether to interleave the pixels colors, meaning keep them in the <paramref name="orderOfExtraction"/> order, or leave them in the plannar form:
Expand All @@ -120,28 +127,25 @@ public static ImagePixelExtractingEstimator ExtractPixels(this TransformsCatalog
bool outputAsFloatArray = ImagePixelExtractingEstimator.Defaults.Convert)
=> new ImagePixelExtractingEstimator(CatalogUtils.GetEnvironment(catalog), outputColumnName, inputColumnName, colorsToExtract, orderOfExtraction, interleavePixelColors, offsetImage, scaleImage, outputAsFloatArray);

/// <include file='doc.xml' path='doc/members/member[@name="ImagePixelExtractingEstimator"]/*' />
/// <summary>
/// Create a <see cref="ImagePixelExtractingEstimator"/>, which exctracts pixel values from the data specified in column: <see cref="ImagePixelExtractingEstimator.ColumnOptions.InputColumnName"/>
/// to a new column: <see cref="ImagePixelExtractingEstimator.ColumnOptions.Name"/>.
/// </summary>
/// <param name="catalog">The transform's catalog.</param>
/// <param name="columnOptions">The <see cref="ImagePixelExtractingEstimator.ColumnOptions"/> describing how the transform handles each image pixel extraction output input column pair.</param>
[BestFriend]
internal static ImagePixelExtractingEstimator ExtractPixels(this TransformsCatalog catalog, params ImagePixelExtractingEstimator.ColumnOptions[] columnOptions)
=> new ImagePixelExtractingEstimator(CatalogUtils.GetEnvironment(catalog), columnOptions);

/// <summary>
/// Resizes the images to a new width and height.
/// Create a <see cref="ImageResizingEstimator"/>, which resize the image from the column specified in <paramref name="inputColumnName"/>
/// to a new column: <paramref name="outputColumnName"/>.
/// </summary>
/// <remarks>
/// In image processing pipelines, often machine learning practitioner make use of<a href= "https://blogs.msdn.microsoft.com/mlserver/2017/04/12/image-featurization-with-a-pre-trained-deep-neural-network-model/">
/// pre-trained DNN featurizers</a> to extract features for usage in the machine learning algorithms.
/// Those pre-trained models have a defined width and height for their input images, so often, after getting loaded, the images will need to get resized before
/// further processing.
/// The new width and height can be specified in the <paramref name="imageWidth"/> and <paramref name="imageHeight"/>
/// <seealso cref = "ImageEstimatorsCatalog" />
/// <seealso cref= "ImageLoadingEstimator" />
/// </remarks >
/// <param name="catalog">The transform's catalog.</param>
/// <param name="outputColumnName">Name of the column resulting from the transformation of <paramref name="inputColumnName"/>.</param>
/// <param name="inputColumnName">Name of column to transform. If set to <see langword="null"/>, the value of the <paramref name="outputColumnName"/> will be used as source.</param>
/// <param name="outputColumnName">Name of the column resulting from the transformation of <paramref name="inputColumnName"/>.
/// This column's data type will be the same as that of the input column.</param>
/// <param name="inputColumnName">Name of the column with images.
/// This estimator operates over <see cref="System.Drawing.Bitmap"/>.</param>
/// <param name="imageWidth">The transformed image width.</param>
/// <param name="imageHeight">The transformed image height.</param>
/// <param name="resizing"> The type of image resizing as specified in <see cref="ImageResizingEstimator.ResizingKind"/>.</param>
Expand Down Expand Up @@ -171,8 +175,6 @@ public static ImageResizingEstimator ResizeImages(this TransformsCatalog catalog
/// further processing.
/// The new width and height, as well as other properties of resizing, like type of scaling (uniform, or non-uniform), and whether to pad the image,
/// or just crop it can be specified separately for each column loaded, through the <see cref="ImageResizingEstimator.ColumnOptions"/>.
/// <seealso cref = "ImageEstimatorsCatalog" />
/// <seealso cref= "ImageLoadingEstimator" />
/// </remarks >
/// <param name="catalog">The transform's catalog.</param>
/// <param name="columnOptions">The <see cref="ImageResizingEstimator.ColumnOptions"/> describing how the transform handles each image resize column.</param>
Expand All @@ -196,13 +198,16 @@ internal static VectorToImageConvertingEstimator ConvertToImage(this TransformsC
=> new VectorToImageConvertingEstimator(CatalogUtils.GetEnvironment(catalog), columnOptions);

/// <summary>
/// Converts vectors of pixels into <see cref="ImageDataViewType"/> representation.
/// Create a <see cref="VectorToImageConvertingEstimator"/>, which creates image from the data from the column specified in <paramref name="inputColumnName"/>
/// to a new column: <paramref name="outputColumnName"/>.
/// </summary>
/// <param name="catalog">The transforms' catalog.</param>
/// <param name="catalog">The transform's catalog.</param>
/// <param name="imageHeight">The height of the output images.</param>
/// <param name="imageWidth">The width of the output images.</param>
/// <param name="outputColumnName"> Name of the column resulting from the transformation of <paramref name="inputColumnName"/>.</param>
/// <param name="inputColumnName"> Name of column to transform. If set to <see langword="null"/>, the value of the <paramref name="outputColumnName"/> will be used as source.</param>
/// <param name="outputColumnName">Name of the column resulting from the transformation of <paramref name="inputColumnName"/>.
/// This column's data type will be <see cref="System.Drawing.Bitmap"/>.</param>
/// <param name="inputColumnName">Name of the column with data to be converted to image.
/// This estimator operates over known-sized vector of <see cref="System.Single"/>, <see cref="System.Double"/> and <see cref="System.Byte"/>.</param>
/// <param name="colorsPresent">Specifies which <see cref="ImagePixelExtractingEstimator.ColorBits"/> are in present the input pixel vectors. The order of colors is specified in <paramref name="orderOfColors"/>.</param>
/// <param name="orderOfColors">The order in which colors are presented in the input vector.</param>
/// <param name="interleavedColors">Whether the pixels are interleaved, meaning whether they are in <paramref name="orderOfColors"/> order, or separated in the planar form:
Expand Down
38 changes: 20 additions & 18 deletions src/Microsoft.ML.ImageAnalytics/ImageGrayscale.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,18 +32,8 @@ namespace Microsoft.ML.Transforms.Image
// REVIEW: Rewrite as LambdaTransform to simplify.
// REVIEW: Should it be separate transform or part of ImageResizerTransform?
/// <summary>
/// <see cref="ITransformer"/> produced by fitting the <see cref="IDataView"/> to an <see cref="ImageGrayscalingEstimator" />.
/// <see cref="ITransformer"/> resulting from fitting an <see cref="ImageGrayscalingTransformer"/>.
/// </summary>
/// <remarks>
/// Calling <see cref="ITransformer.Transform(IDataView)"/> converts the image to grayscale.
/// The images might be converted to grayscale to reduce the complexity of the model.
/// The grayed out images contain less information to process than the colored images.
/// Another use case for converting to grayscale is to generate new images out of the existing ones, so you can have a larger dataset,
/// a technique known as <a href = "http://www.stat.harvard.edu/Faculty_Content/meng/JCGS01.pdf"> data augmentation</a>.
/// For end-to-end image processing pipelines, and scenarios in your applications, see the
/// <a href="https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/getting-started"> examples in the machinelearning-samples github repository.</a>
/// <seealso cref = "ImageEstimatorsCatalog" />
/// </remarks>
public sealed class ImageGrayscalingTransformer : OneToOneTransformerBase
{
internal sealed class Column : OneToOneColumn
Expand Down Expand Up @@ -226,21 +216,33 @@ protected override Delegate MakeGetter(DataViewRow input, int iinfo, Func<int, b
}

/// <summary>
/// <see cref="IEstimator{TTransformer}"/> that converts the image to grayscale.
/// <see cref="IEstimator{TTransformer}"/> for the <see cref="ImageGrayscalingTransformer"/>.
/// </summary>
/// <remarks>
/// Calling <see cref="IEstimator{TTransformer}.Fit(IDataView)"/> in this estimator, produces an <see cref="ImageGrayscalingEstimator"/>.
/// <format type="text/markdown"><![CDATA[
/// ### Estimator Characteristics
/// | | |
/// | -- | -- |
/// | Does this estimator need to look at the data to train its parameters? | No |
/// | Input column data type | <xref:System.Drawing.Bitmap> |
/// | Output column data type | <xref:System.Drawing.Bitmap> |
/// | Required NuGet in addition to Microsoft.ML | Microsoft.ML.ImageAnalytics |
///
/// The resulting <xref:Microsoft.ML.Transforms.Image.ImageGrayscalingTransformer> creates a new column, named as specified in the output column name parameters, and
/// converts the image from the input column into a grayscale image.
/// The images might be converted to grayscale to reduce the complexity of the model.
/// The grayed out images contain less information to process than the colored images.
/// Another use case for converting to grayscale is to generate new images out of the existing ones, so you can have a larger dataset,
/// a technique known as <a href = "http://www.stat.harvard.edu/Faculty_Content/meng/JCGS01.pdf"> data augmentation</a>.
/// a technique known as [data augmentation](http://www.stat.harvard.edu/Faculty_Content/meng/JCGS01.pdf).
/// For end-to-end image processing pipelines, and scenarios in your applications, see the
/// <a href="https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/getting-started"> examples in the machinelearning-samples github repository.</a>
/// <seealso cref = "ImageEstimatorsCatalog" />
/// </remarks >
/// [examples](https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/getting-started) in the machinelearning-samples github repository.
/// See the See Also section for links to more examples of the usage.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

examples of the usage [](start = 51, length = 21)

@natke usage examples?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that sounds better

/// ]]>
/// </format>
/// </remarks>
/// <seealso cref="ImageEstimatorsCatalog.ConvertToGrayscale(TransformsCatalog, string, string)" />
public sealed class ImageGrayscalingEstimator : TrivialEstimator<ImageGrayscalingTransformer>
{

/// <summary>
/// Converts the images to grayscale.
/// </summary>
Expand Down
Loading