ML.NET and Unity

[mihaar]

System information

• win 10:
• .NET Version ( .NET Global Assembly Cache Utility. Version 4.0.30319.0):

Issue

Unity editor reports two errors:
1.Assembly 'Library/ScriptAssemblies/Assembly-CSharp.dll' will not be loaded due to errors:
Reference has errors 'MLAICore'.
2.Assembly 'Assets/Plugins/MLAICore.dll' will not be loaded due to errors:
Unable to resolve reference 'Microsoft.ML.Data'. Is the assembly missing or incompatible with the current platform?

• What did you do?
  I have made an API that loads model.zip file and predicts a nr given some input values(It is almost exactly as your taxi fare example). Through commenting out lines and rebuilding my api I deduced that if I put this code: MLContext ml = new MLContext(seed: 0); into any of my api methods I get this error when importing the api into unity.
• What happened?
  It all works great but when I import this api into Unity, the unity editor returns an error.
• What did you expect?
  Since unity has now support for .net 2.0 apis and frameworks 4+ I was hoping ML.Net would work in that environment. Any sugestions would be most welcom.
  sampleUnityProject.zip
  exampleDotnetApiProject.zip
  https://stackoverflow.com/questions/53781312/ml-net-in-unity/53781387#comment94434358_53781312

Source code / logs

using System; using Microsoft.ML; namespace MLAICore { public class Class1 { public static string SayHello() { MLContext ml = new MLContext(seed: 0);//if I comment out this line code compiles in unity return "hello from core ML"; //and i get this response, if I live it as is =>err } } }

Please paste or attach the code or logs or traces that would be helpful to diagnose the issue you are reporting.

[rogancarr]

Hi @mihaar ,

Thanks for filing this issue! Technically, ML.NET should work with Unity, but we haven't done comprehensive testing for Unity yet. We've added this for the 0.10 schedule (starting early January, releasing at the end of January).

One question: What version of Unity are you using?

[fashrista]

..I am using unity version: 2018.3.0f2 Personal.
And I feel obligated to add that I am not quite sure of what I am doing, but I also find it impossible to build a class library that has it’s ML.Net dependencies included, so I could than use this lib in , let’s say .net console app.
Also, if I just add Microsoft.ML dlls to unity plugins folder I also get the same errors ... I am a beginner but if I can help in any way I would be happy to.
ps.:I have posted this issue under mihaar username
@shauheen
AN UPDATE:
What did you do?
1.Unity setup: player settings/standalone/ScriptingRuntimeVersion: .NET 4.x Equivalent, player settings/standalone/APICompatibilityLevel: .NET 4.x Equivalent
2. Made a folder: "MyUnityProject"\Assets\Plugins\ML and pasted all ML .NET dlls into it (got them on gradle). Unity excepted them with no errors.
3. Have "copied" iris example from https://dotnet.microsoft.com/learn/machinelearning-ai/ml-dotnet-get-started-tutorial
What happened?
4.Have trained succsessfully (I think) but, when trying to make a prediction mlModel.MakePredictionFunction it all goes bad with ArgumentException: method arguments are incompatible error as listed below.
What did you expect?
A regression prediction as I get in VisualStudio solution by doing 3. and 4.

[longgood]

looking forward to see it works with Unity. and announce sooner in the end of coming Jan. Though we are now seeking for the tensorflow complied with Unity.

[mihaar]

@rogancarr I have managet to setup unity in a way I can now import ML dlls and train models, but when trying to make a prediction I came across this error: (similar code works in .net framework 4.7.2 solution)
line var predictionFunction = model.MakePredictionFunction<AbstractSignal, PredictedRfd>(mlContext); returns>
ArgumentException: method arguments are incompatible
System.Delegate.CreateDelegate (System.Type type, System.Object firstArgument, System.Reflection.MethodInfo method, System.Boolean throwOnBindFailure, System.Boolean allowClosed) (at :0)
System.Delegate.CreateDelegate (System.Type type, System.Reflection.MethodInfo method, System.Boolean throwOnBindFailure) (at :0)
System.Delegate.CreateDelegate (System.Type type, System.Reflection.MethodInfo method) (at :0)
System.Reflection.Emit.DynamicMethod.CreateDelegate (System.Type delegateType) (at :0)
Microsoft.ML.Runtime.Api.ApiUtils.GeneratePeek[TOwn,TRow,TValue] (System.Reflection.FieldInfo fieldInfo, System.Reflection.Emit.OpCode assignmentOpCode) (at :0)
System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at :0)
Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at :0)
System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at :0)
Microsoft.ML.Runtime.Api.ApiUtils.GeneratePeek[TOwn,TRow] (Microsoft.ML.Runtime.Api.InternalSchemaDefinition+Column column) (at :0)
Microsoft.ML.Runtime.Api.DataViewConstructionUtils+DataViewBase1[TRow]..ctor (Microsoft.ML.Runtime.IHostEnvironment env, System.String name, Microsoft.ML.Runtime.Api.InternalSchemaDefinition schemaDefn) (at <e71d766983cd4fc888a6692d75dfd109>:0) Microsoft.ML.Runtime.Api.DataViewConstructionUtils+ListDataView1[TRow]..ctor (Microsoft.ML.Runtime.IHostEnvironment env, System.Collections.Generic.IList1[T] data, Microsoft.ML.Runtime.Api.InternalSchemaDefinition schemaDefn) (at <e71d766983cd4fc888a6692d75dfd109>:0) Microsoft.ML.Runtime.Api.DataViewConstructionUtils.CreateFromList[TRow] (Microsoft.ML.Runtime.IHostEnvironment env, System.Collections.Generic.IList1[T] data, Microsoft.ML.Runtime.Api.SchemaDefinition schemaDefinition) (at :0)
Microsoft.ML.Runtime.Api.ComponentCreation.CreateDataView[TRow] (Microsoft.ML.Runtime.IHostEnvironment env, System.Collections.Generic.IList1[T] data, Microsoft.ML.Runtime.Api.SchemaDefinition schemaDefinition) (at <e71d766983cd4fc888a6692d75dfd109>:0) Microsoft.ML.Runtime.Data.PredictionFunction2[TSrc,TDst]..ctor (Microsoft.ML.Runtime.IHostEnvironment env, Microsoft.ML.Core.Data.ITransformer transformer) (at :0)
Microsoft.ML.Runtime.Data.PredictionFunctionExtensions.MakePredictionFunction[TSrc,TDst] (Microsoft.ML.Core.Data.ITransformer transformer, Microsoft.ML.Runtime.IHostEnvironment env) (at :0)
MLaiTest.TestSinglePrediction (System.Collections.Generic.List`1[T] signal, Microsoft.ML.MLContext mlContext, System.String modelPath) (at Assets/MLaiTest.cs:131)
MLaiTest.Start () (at Assets/MLaiTest.cs:28)

Example is attached
newTestCanMlaiDELETE.zip

[fashrista]

Don't know if this helps, but the iris example in unity returns the same error: ArgumentException: method arguments are incompatible.
What I did to Imprt ML.NET dlls intu Unity: https://stackoverflow.com/questions/53781312/ml-net-in-unity/53781387#53781387

[mihaar]

As sugested by @shauheen I have tryed importing into Unity and training ML version 0.9.0. The cal: predictionEngine = model.CreatePredictionEngine<AbstractSignal, PredictedRfd>(mlContext); now returns the same error:
ArgumentException: method arguments are incompatible
System.Delegate.CreateDelegate (System.Type type, System.Object firstArgument, ...

(mind you, method MakePredictionFunction is now renamed to CreatePredictionEngine, but it seems the error is the same)

[mihaar]

EDIT: Aivaras Ivoškus from Unity says: "The issue does not seem to be related to Scripting Restrictions of Unity Engine. "

@shauheen @rogancarr @danmosemsft UNITY SCRIPTING RESTRICTIONS
, says that:

System.Reflection.Emit

An AOT platform cannot implement any of the methods in the System.Reflection.Emit namespace. Note that the rest of System.Reflection is acceptable, as long as the compiler can infer that the code used via reflection needs to exist at runtime.

I think the method Microsoft.ML.ApiUtils.GeneratePeek[TOwn,TRow] (Microsoft.ML.Data.InternalSchemaDefinition+Column column) (at <22a400311822408a8bd05691f3cdc119>:0) does that exactly, or am I wrong?