Rename DataView Metadata to Annotations.

Fix dotnet#1843
Fix dotnet#2297

Author: eerhardt

docs/code/IDataViewDesignPrinciples.md

@@ -47,7 +47,7 @@ only when needed to satisfy a local request for information.
 The IDataView design fulfills the following design requirements:
 
 * **General schema**: Each view carries schema information, which specifies
-  the names and types of the view's columns, together with metadata associated
+  the names and types of the view's columns, together with annotations associated
   with the columns. The system is optimized for a reasonably small number of
   columns (hundreds). See [here](#basics).
 
@@ -112,14 +112,14 @@ The IDataView system design does *not* include the following:
 * **Multi-view schema information**: There is no direct support for specifying
   cross-view schema information, for example, that certain columns are primary
   keys, and that there are foreign key relationships among tables. However,
-  the column metadata support, together with conventions, may be used to
+  the column annotation support, together with conventions, may be used to
   represent such information.
 
 * **Standard ML schema**: The IDataView system does not define, nor prescribe,
   standard ML schema representation. For example, it does not dictate
   representation of nor distinction between different semantic interpretations
   of columns, such as label, feature, score, weight, etc. However, the column
-  metadata support, together with conventions, may be used to represent such
+  annotation support, together with conventions, may be used to represent such
   interpretations.
 
 * **Row count**: A view is not required to provide its row count. The
@@ -149,7 +149,7 @@ The IDataView system design does *not* include the following:
 
 IDataView has general schema support, in that a view can have an arbitrary
 number of columns, each having an associated name, index, data type, and
-optional metadata.
+optional annotation.
 
 Column names are case sensitive. Multiple columns can share the same name, in
 which case, one of the columns hides the others, in the sense that the name
@@ -177,7 +177,7 @@ The set of standard types will likely be expanded over time.
 The IDataView type system is specified in a separate document, *IDataView Type
 System Specification*.
 
-IDataView provides a general mechanism for associating semantic metadata with
+IDataView provides a general mechanism for associating semantic annotations with
 columns, such as designating sets of score columns, names associated with the
 individual slots of a vector-valued column, values associated with a key type
 column, whether a column's data is normalized, etc.

docs/code/IDataViewImplementation.md

@@ -313,10 +313,10 @@ are initialized using the pseudo-random number generator in an `IHost` that
 changes from one to another. But, that's a bit nit-picky.
 
 Note also: when we say functionally identical we include everything about it:
-not just the data, but the schema, its metadata, the implementation of
+not just the data, but the schema, its annotations, the implementation of
 shuffling, etc. For this reason, while serializing the data *model* has
 guarantees of consistency, serializing the *data* has no such guarantee: if
-you serialize data using the text saver, practically all metadata (except slot
+you serialize data using the text saver, practically all annotations (except slot
 names) will be completely lost, which can have implications on how some
 transforms and downstream processes work. Or: if you serialize data using the
 binary saver, suddenly it may become shufflable whereas it may not have been
@@ -475,7 +475,7 @@ helpful).
 
 The schema contains information about the columns. As we see in [the design
 principles](IDataViewDesignPrinciples.md), it has index, data type, and
-optional metadata.
+optional annotations.
 
 While *programmatically* accesses to an `IDataView` are by index, from a
 user's perspective the indices are by name; most training algorithms
@@ -498,20 +498,20 @@ things like key-types and vector-types, when returned, should not be created
 in the function itself (thereby creating a new object every time), but rather
 stored somewhere and returned.
 
-## Metadata
+## Annotations
 
-Since metadata is *optional*, one is not obligated to necessarily produce it,
+Since annotations are *optional*, one is not obligated to necessarily produce it,
 or conform to any particular schemas for any particular kinds (beyond, say,
 the obvious things like making sure that the types and values are consistent).
 However, the flip side of that freedom given to *producers*, is that
 *consumers* are obligated, when processing a data view input, to react
-gracefully when metadata of a certain kind is absent, or not in a form that
-one expects. One should *never* fail when input metadata is in a form one does
+gracefully when an annotation of a certain kind is absent, or not in a form that
+one expects. One should *never* fail when input annotations are in a form one does
 not expect.
 
 To give a practical example of this: many transforms, learners, or other
 components that process `IDataView`s will do something with the slot names,
-but when the `SlotNames` metadata kind for a given column is either absent,
+but when the `SlotNames` annotation kind for a given column is either absent,
 *or* not of the right type (vectors of strings), *or* not of the right size
 (same length vectors as the input), the behavior is not to throw or yield
 errors or do anything of the kind, but to simply say, "oh, I don't really have

docs/code/IDataViewTypeSystem.md

@@ -63,7 +63,7 @@ components. At a high level, it is analogous to the .Net interface
 While `IEnumerable<T>` is a sequence of objects of type `T`, `IDataView` is a
 sequence of rows. An `IDataView` object has an associated `ISchema` object
 that defines the `IDataView`'s columns, including their names, types, indices,
-and associated metadata. Each row of the `IDataView` has a value for each
+and associated annotations. Each row of the `IDataView` has a value for each
 column defined by the schema.
 
 Just as `IEnumerable<T>` has an associated enumerator interface, namely
@@ -224,29 +224,29 @@ to a dense representation having the suppressed entries filled in with the
 entries are emphatically *not* the missing/`NA` value of the item type, unless
 the missing and default values are identical, as they are for key types.
 
-### Metadata
+### Annotations
 
-A column in an `ISchema` can have additional column-wide information, known as
-metadata. For each string value, known as a metadata kind, a column may have a
-value associated with that metadata kind. The value also has an associated
+A column in an `DataViewSchema` can have additional column-wide information, known as
+annotations. For each string value, known as an annotation kind, a column may have a
+value associated with that annotation kind. The value also has an associated
 type, which is a compatible column type.
 
 For example:
 
 * A column may indicate that it is normalized, by providing a `BL` valued
-  piece of metadata named `IsNormalized`.
+  annotation named `IsNormalized`.
 
 * A column whose type is `V<R4,17>`, meaning a vector of length 17 whose items
-  are single-precision floating-point values, might have `SlotNames` metadata
+  are single-precision floating-point values, might have `SlotNames` annotation
   of type `V<TX,17>`, meaning a vector of length 17 whose items are text.
 
 * A column produced by a scorer may have several pieces of associated
-  metadata, indicating the "scoring column group id" that it belongs to, what
+  annotations, indicating the "scoring column group id" that it belongs to, what
   kind of scorer produced the column (for example, binary classification), and the
   precise semantics of the column (for example, predicted label, raw score,
   probability).
 
-The `ISchema` interface, including the metadata API, is fully specified in
+The `DataViewSchema` class, including the annotations API, is fully specified in
 another document.
 
 ## Text Type

docs/code/MlNetHighLevelConcepts.md

@@ -29,9 +29,9 @@ This document is going to cover the following ML.NET concepts:
 In ML.NET, data is very similar to a SQL view: it's a lazily-evaluated, cursorable, heterogenous, schematized dataset.
 
 - It has *Schema* (an instance of an `ISchema` interface), that contains the information about the data view's columns.
-  - Each column has a *Name*, a *Type*, and an arbitrary set of *metadata* associated with it.
+  - Each column has a *Name*, a *Type*, and an arbitrary set of *annotations* associated with it.
   - It is important to note that one of the types is the `vector<T, N>` type, which means that the column's values are *vectors of items of type T, with the size of N*. This is a recommended way to represent multi-dimensional data associated with every row, like pixels in an image, or tokens in a text.
-  - The column's *metadata* contains information like 'slot names' of a vector column and suchlike. The metadata itself is actually represented as another one-row *data*, that is unique to each column.
+  - The column's *annotations* contains information like 'slot names' of a vector column and suchlike. The annotations itself are actually represented as another one-row *data*, that is unique to each column.
 - The data view is a source of *cursors*. Think SQL cursors: a cursor is an object that iterates through the data, one row at a time, and presents the available data.
   - Naturally, data can have as many active cursors over it as needed: since data itself is immutable, cursors are truly independent.
   - Note that cursors typically access only a subset of columns: for efficiency, we do not compute the values of columns that are not 'needed' by the cursor.

docs/code/SchemaComprehension.md

@@ -8,17 +8,17 @@ For a better understanding of `IDataView` principles and type system please refe
 
 ## Introduction
 
-Every dataset in ML.NET is represented as an `IDataView`, which is, for the purposes of this document, a collection of rows that share the same columns. The set of columns, their names, types and other metadata is known as the *schema* of the `IDataView`, and it's represented as an `ISchema` object.
+Every dataset in ML.NET is represented as an `IDataView`, which is, for the purposes of this document, a collection of rows that share the same columns. The set of columns, their names, types and other annotations is known as the *schema* of the `IDataView`, and it's represented as an `ISchema` object.
 
 In this document, we will be using the terms *data view* and `IDataView` interchangeably, same for *schema* and `ISchema`.
 
 Before any new data enters ML.NET, the user needs to somehow define how the schema of the data will look like.
 To do this, the following questions need to be answered:
 - What are the column names?
 - What are their types?
-- What other metadata is associated with the columns?
+- What other annotations are associated with the columns?
 
-These items above are very similar to the definition of fields in a C# class: names and types of columns correspond to names and types of fields, and metadata can correspond to field attributes. 
+These items above are very similar to the definition of fields in a C# class: names and types of columns correspond to names and types of fields, and annotations can correspond to field attributes. 
 Because of this similarity, ML.NET offers a common convenient mechanism for creating a schema: it is done via defining a C# class.
 
 For example, the below class definition can be used to define a data view with 5 float columns:
@@ -201,10 +201,10 @@ var dataView = env.CreateDataView<IrisVectorData>(arr, schemaDef);
 var predictionEngine = env.CreatePredictionEngine<IrisData, IrisVectorData>(dv, outputSchemaDefinition: schemaDef);
 ```
 
-In addition to the above, you can use `SchemaDefinition` to add per-column metadata:
+In addition to the above, you can use `SchemaDefinition` to add per-column annotations:
 ```C#
-// Add column metadata.
-schemaDef["Label"].AddMetadata(MetadataUtils.Kinds.HasMissingValues, false);
+// Add column annotation.
+schemaDef["Label"].AddAnnotation(MetadataUtils.Kinds.HasMissingValues, false);
 ```
 
 ## Limitations
@@ -216,7 +216,7 @@ Here is the list of things that are only possible via the low-level interface:
 * Creating or reading a data view, where even column *types* are not known at compile time (so you cannot create a C# class to define the schema)
   * This can happen if you write a general-purpose machine learning tool that can ingest different kinds of datasets.
 * Reading a subset of columns that differs from one row to another: the cursor always populates the entire row object.
-* Reading column metadata from the data view.
+* Reading column annotations from the data view.
 * Accessing the 'hidden' data view columns by index. 
   * Hidden columns are those that have the same name as other columns and a smaller index. They are not accessible by name.
 * Creating 'cursor sets': this is a feature that lets you iterate over data in multiple parallel threads by splitting the data between multiple 'sibling' cursors.

docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/MulticlassClassification/LightGBMMulticlassClassification.cs

@@ -56,9 +56,9 @@ public static void Example()
             // IDataView with predictions, to an IEnumerable<DatasetUtils.MulticlassClassificationExample>.
             var nativePredictions = mlContext.CreateEnumerable<DatasetUtils.MulticlassClassificationExample>(dataWithPredictions, false).ToList();
 
-            // Get schema object out of the prediction. It contains metadata such as the mapping from predicted label index
+            // Get schema object out of the prediction. It contains annotations such as the mapping from predicted label index
             // (e.g., 1) to its actual label (e.g., "AA").
-            // The metadata can be used to get all the unique labels used during training.
+            // The annotations can be used to get all the unique labels used during training.
             var labelBuffer = new VBuffer<ReadOnlyMemory<char>>();
             dataWithPredictions.Schema["PredictedLabelIndex"].GetKeyValues(ref labelBuffer);
             // nativeLabels is { "AA" , "BB", "CC", "DD" }

docs/samples/Microsoft.ML.Samples/Static/LightGBMMulticlassWithInMemoryData.cs

@@ -70,15 +70,15 @@ public void MultiClassLightGbmStaticPipelineWithInMemoryData()
             // Convert prediction in ML.NET format to native C# class.
             var nativePredictions = mlContext.CreateEnumerable<DatasetUtils.MulticlassClassificationExample>(prediction.AsDynamic, false).ToList();
 
-            // Get schema object out of the prediction. It contains metadata such as the mapping from predicted label index
+            // Get schema object out of the prediction. It contains annotations such as the mapping from predicted label index
             // (e.g., 1) to its actual label (e.g., "AA"). The call to "AsDynamic" converts our statically-typed pipeline into
-            // a dynamically-typed one only for extracting metadata. In the future, metadata in statically-typed pipeline should
+            // a dynamically-typed one only for extracting annotations. In the future, annotations in statically-typed pipeline should
             // be accessible without dynamically-typed things.
             var schema = prediction.AsDynamic.Schema;
 
             // Retrieve the mapping from labels to label indexes.
             var labelBuffer = new VBuffer<ReadOnlyMemory<char>>(); 
-            schema[nameof(DatasetUtils.MulticlassClassificationExample.PredictedLabelIndex)].Metadata.GetValue("KeyValues", ref labelBuffer);
+            schema[nameof(DatasetUtils.MulticlassClassificationExample.PredictedLabelIndex)].Annotations.GetValue("KeyValues", ref labelBuffer);
             // nativeLabels is { "AA" , "BB", "CC", "DD" }
             var nativeLabels = labelBuffer.DenseValues().ToArray(); // nativeLabels[nativePrediction.PredictedLabelIndex - 1] is the original label indexed by nativePrediction.PredictedLabelIndex.