Skip to content

TensorFlowUtils doesn't really fit with the 1.0 API #2572

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rogancarr opened this issue Feb 15, 2019 · 1 comment · Fixed by #2672
Closed

TensorFlowUtils doesn't really fit with the 1.0 API #2572

rogancarr opened this issue Feb 15, 2019 · 1 comment · Fixed by #2672
Assignees
Labels
API Issues pertaining the friendly API question Further information is requested
Milestone

Comments

@rogancarr
Copy link
Contributor

rogancarr commented Feb 15, 2019

One of the scenarios we want to support in ML.NET 1.0 is the ability to examine the schema of a TensorFlow model (see example below [1]). To do this, we currently use helper methods in the TensorFlowUtils class [2].

Right now, this class and these methods are public and can be used as part of an ML.NET program, but that doesn't seem like the way we want to go in v1.0. What do you all think? Does it make sense to add these methods to mlContext and make them internal/BestFriend? If we add them to a catalog, where should they go?

[1] Example of why you would want to examine the schema before loading. Here, we want to get the layer names and their sizes before specifying them as outputs in our IDataView:

// Get the output size of a layer
var schema = TensorFlowUtils.GetModelSchema(mlContext, model_location);
var outputColumn = schema.GetColumnOrNull("Output");
if (outputColumn.HasValue)
    outputLength = ((VectorType)outputColumn.Value.Type).Dimensions[0];

[2] Helper Methods

public static Schema GetModelSchema(IHostEnvironment env, string modelPath);
public static IEnumerable<(string, string, ColumnType, string[])> GetModelNodes(IHostEnvironment env, string modelPath);
public static TensorFlowModelInfo LoadTensorFlowModel(IHostEnvironment env, string modelPath);

where LoadTensorFlowModel is already in the Transforms catalog.

Would GetModelSchema and GetModelNodes go in the DataOperations catalog?

And to pile on, GetModelNodes (which may not be necessary to expose) returns a list of returns, instead of an object, which we definitely can't have in V1.

Related to #2498

@rogancarr rogancarr added question Further information is requested API Issues pertaining the friendly API labels Feb 15, 2019
@Ivanidzo4ka
Copy link
Contributor

Related to #2552.
I think main discussion is going on in that issue.

@zeahmed zeahmed self-assigned this Feb 15, 2019
@shauheen shauheen added this to the 0219 milestone Feb 21, 2019
@ghost ghost locked as resolved and limited conversation to collaborators Mar 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
API Issues pertaining the friendly API question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants