Retrain Custom TensorFlow Model Feature Request

[LaurensVanAcker]

I designed (not trained) a siamese CNN network in Keras (python) and exported it as a SavedModel with the random weights.
_Input = 4 dimensions (left/right image, height, width, RGB).
Output = change of images being equal (sigmoid)

But I don't understand how I can train this model from .NET Core?

When I use:

using var tensorFlowModel = mlContext.Model.LoadTensorFlowModel("./model");
var pipeline = tensorFlowModel.ScoreTensorFlowModel(
 new[] { inputComparer.Name },
 new[] { outputComparer.Name }, addBatchDimensionInput: true);

Then I potentially can get the output from the model, but it's untrained, so this makes no sense.
So I need a trainer. I tried with LbfgsLogisticRegression but then I get "Schema mismatch for label column 'StatefulPartitionedCall': expected Boolean, got Vector" which makes sense as the Tensorflow model don't output Booleans, but a Vector. Do I need another trainer?

When I use the ImageClassification from this tutorial, I can only train one of the embedded networks from ImageClassificationTrainer.Architecture. It's a class output instead of binary, but that's OK. One class is also binary. But I want to train my own network.

This example was the closest I could find.

How do I train my binary output Tensorflow siamese network?

The reason I want to train this network from .NET Core instead of Python is to test the speed improvements and rollout something independently of Python.

[chriss2401]

Not sure if this is the right answer, but you can also try using Tensorflow.NET for this purpose (what mlnet builds upon):
https://github.com/SciSharp/TensorFlow.NET

[mstfbl]

Hi @LaurensVanAcker,

To my knowledge, you currently cannot retrain another model other than one of the embeded networks from ImageClassificationTrainer.Architecture (as you've noted) on ML.NET. This can be a feature request which can be worked on in the future.

[LaurensVanAcker]

I think it's an advantage if it could be trained from within Dot NET, so you can retrain an existing custom model in production using the existing input (if the statistical variables are the same). Data also needs a lot of preprocessing. So this means I now need to build this preprocessing workflow both in Python as in C#. And data augmentation works completely differently in both.

Somebody already started this, but there are no new commits?
It's listed in 15.1 previews (not downloadable?) but I can't see commits to it. So I think it's wrong.

https://github.com/dotnet/machinelearning/blob/master/src/Microsoft.ML.Vision/DnnRetrainTransform.cs
https://github.com/dotnet/machinelearning/blob/f7aaa5a410992ba89ba08bca234fc2814ed5c6d3/src/Microsoft.ML.Vision/VisionCatalog.cs
https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.dnncatalog.retraindnnmodel?view=ml-dotnet-preview-0.15.1
#4520

https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.tensorflowcatalog?view=ml-dotnet

I spend quite some time on this as the documentation currently indicates it's possible:

"Retraining of TensorFlow model: In this mode, the transform retrains a TensorFlow model using 
the user data passed through ML.Net pipeline. Once the model is trained,
 it's outputs can be used as features for scoring."