Introduce DenseVector<T> and convert one usage of VBuffer to it.

[eerhardt]

Working towards dotnet#608

This is an initial draft of what a "dense vector" could look like in ML.NET. I also updated one location to use it to show how it would work.

Please let me know if you think I'm going in the wrong direction.

@TomFinley @ericstj @Zruty0 @KrzysztofCwalina

[KrzysztofCwalina]

Why do we need this type? Why not use Memory directly? If you need to be able to resize it, could the caller just hold to the original buffer? Or the method filling in data into Memory would return out parameter specifying how many items were filled in.

[eerhardt]

Take a look at https://github.com/dotnet/machinelearning/blob/master/docs/code/VBufferCareFeeding.md#buffer-re-use-as-a-user for the common usage of these vectors.

Why not use Memory directly?

Because if you only had Memory directly, as soon as you wanted to return a vector that had a length less than the total buffer capacity, now you could no longer get back to use the total capacity of the buffer.

could the caller just hold to the original buffer?

It's my understanding that the caller doesn't always know how big the original buffer should be, and thus can't create it.

Or the method filling in data into Memory would return out parameter specifying how many items were filled in.

That won't work with the current IDataView GetGetter delegate design.

[KrzysztofCwalina]

What I meant is the following (it's a pattern we use in lots of Span/Memory APIs):

Memory<byte> buffer = ...
while(true){
    if(TryFillBuffer(buffer, out written)) {
        UseFilledInBuffer(buffer.Slice(0, written));
        break;
    }
    Enlarge(ref buffer);  
} 

Could we do something like that?

[KrzysztofCwalina]

Will the type show in public surface area of mainline scenario APIs? If yes, should this type be in Microsoft.ML root?

[KrzysztofCwalina]

Why do we need these helpers?

[KrzysztofCwalina]

I assume these out.txt files should not be here?

[eerhardt]

After speaking with @TomFinley, this is not the direction we want to go with VBuffer. Closing.