ComponentCatalog design issues

[jkotas]

https://github.com/dotnet/machinelearning/blob/master/src/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs

• Enumerates all types in all loaded assemblies. This pattern is known to have poor performance characteristics that lead to long startup time.
• Enumerates assemblies in application directory. This is not compatible with .NET Core app model. The app assemblies are not guaranteed to be in the application directory in .NET Core (e.g. they can be in one of the shared frameworks or in the assembly cache), or they may not exist at all (single file .exes planned for .NET Core, or .NET Native used for UWP apps).
• A long list of hardcoded names to skip while enumerating:

  machinelearning/src/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs

  Line 306 in c023727

  public static string[] FilePrefixesToAvoid = new string[] {

Does ML.NET really need its own dependency injection framework? Would it be worth looking at decoupling the dependency injection from the ML.Core, and ideally using one of the existing dependency injection frameworks instead of inventing yet another one?

[TomFinley]

So what do you think, MEF, or other?

Over the years we've toyed with the idea of replacing the scheme with MEF but never quite got beyond the "yeah might be nice some day" vagueness. @veikkoeeva mentioned the same idea in #136 actually... It has basically the same idea: multiple implementations of types associated with loadnames, even has a so-called DirectoryCatalog for cases where we want to keep the current system, allows other more explicit schemes for cases where you don't.

[jkotas]

So what do you think, MEF, or other?

My top concern is to make it possible to use ML.Core without heavy-weight dependency injection frameworks.

I do not have opinion on which dependency injection framework is the right one. I believe that people typically like to have an option to bring their own.

[TomFinley]

I see this is your first issue. Welcome @jkotas ! Sorry, I thought I understood the request, but I see I don't. The usage of a some sort of dependency injection framework is a pretty core central concept to how the entire system is able to work at all. Even in just the code that's in this repo so far, I see some hundreds of components advertising themselves as a loadable component and roughly one hundred calls to the component catalog in one place or another to instantiate the corresponding components; and we haven't even published the command line or GUI interfaces yet. I mean, let's just start with model loading. What's your plan?

[jkotas]

I do not have a good feel for the overall ML.Core state yet, so I do not have a good detailed answer to your question.

I think the problem is similar to ASP.NET Core: The lower layers (building blocks) of ASP.NET Core do not have a hard dependency of dependency injection frameworks. The dependency injection frameworks come into play only for the higher levels of ASP.NET Core like MVC. If people want to use ASP.NET without paying the price for the dependency injection frameworks, they can still do that just fine.

@KrzysztofCwalina What do you think about the ML.Core being joined at the hip with a dependency injection framework (its own custom one currently)?

[TomFinley]

Ah OK. Well, again, welcome, and I hope you'll spend some time getting to know the structure of the code a bit more. Let any of us know if you have questions. While I'm sure many libraries (including among them ASP.NET) don't benefit from dependency injection, this library does.

[KrzysztofCwalina]

Yes, I in general worry that ML.NET has too many potentially independent technologies joined at the hip. I talked to @glebuk about it and we agreed to work on simplifying ML.Core.

[glebuk]

ASP.NET and ML.NET are a bit different - the entire ML.NET section of the framework that deals with training and inference relies on having hundreds of independent components (trainers, transforms, loaders, loss functions and so forth) We want to have runtime component binding within the framework.

We should be able to drop a DLL and be able to use it, without having to recompile the common infra. The component catalog allows us to find and load needed components, it also allows us to do customization of a given ML deployment where only needed DLLs are placed.

Moreover, it is a component that allows a runtime binding for higher level APIs, such as GUIs, command line tools and language bindings.

Parts of the framework that do not deal with any components (IDV and related) do not need this, however any time you create a pipeline or try to use any components in any way you would need to load and find them. This is at a pretty low level of the overall architecture.
How should we achieve those goals if we don't have a DI/TLC Component Catalog?

[jkotas]

The abstract concept of component catalog is ok. It is https://en.wikipedia.org/wiki/Service_locator_pattern with all its advantages and disadvantages.

The part that is not good is the messy hardcoded algorithm that populates the component catalog. I think that the best way to fix it would be do add explicit methods to register set of components that you would call at the start of the application. For example, the standard learners package can have method like:

public static class StandardLearnersExtensions
{
    public static ComponentCatalog AddStandardLearners(ComponentCatalog componentCatalog);
}

And one would register them by calling this method at the start of the application:

   componentCatalog.AddStandardLearners()

If you would like to keep the model where all services that happen to be present in the app are registered, it should be opt-in by calling an extension method like componentCatalog.AddComponentsByConvention() at the start of the application. And the implementation of AddComponentsByConvention should get the list of the assemblies that the application is composed from by calling proper APIs, not by enumerating files in a directory. The natural way to add .dlls to .NET Core app is by adding their package reference to the app .csproj file, not by manually dropping the .dll to the app directory.

Enumerating files in a directory and applying ad-hoc filter to them is not compatible with .NET Core and other .NET app models. I would not include this option in MI.NET. If you really need it to keep it around for existing users, I would move it into a obsoleted compat library that folks do not get by default.

[jkotas]

If it helps, here is an example of how ASP.NET Core implements similar concept: https://github.com/aspnet/Mvc-Private/blob/34e4fbf92df2d181b441bbbde598b68d4f33d8b4/src/Microsoft.AspNetCore.Mvc.Formatters.Json/DependencyInjection/MvcJsonMvcCoreBuilderExtensions.cs#L15
These extension methods return its argument to make it easy to chain them with other similar methods. Typical ASP.NET apps have code like this on the initialization path:

services
    .AddMvcCore()
    .AddJsonFormatters()
    .AddSignalR()

[dsyme]

@jkotas @KrzysztofCwalina I've given some feedback on how the ComponentCatalog is causing pain for F# scripting here: #600 (comment). It is also the root cause of #401 which required the awkward workaround of forcing artificial loads of some referenced assemblies:

let _load = 
    [ typeof<Microsoft.ML.Runtime.Transforms.TextAnalytics>
      typeof<Microsoft.ML.Runtime.FastTree.FastTree> ]

It is important that ML.NET work cleanly in conjunction with REPL scripting contexts (F# or C# and future possible REPL-hosted scripting in workbooks etc.). The use of DI and service-locator patterns generally makes this more painful, at least in conjunction to "normal" .NET libraries.

@Ivanidzo4ka and @eerhardt have replied with current thinking at #600 (comment)

We should be able to drop a DLL and be able to use it, without having to recompile the common infra. The component catalog allows us to find and load needed components, it also allows us to do customization of a given ML deployment where only needed DLLs are placed.

Moreover, it is a component that allows a runtime binding for higher level APIs, such as GUIs,
command line tools and language bindings.

Parts of the framework that do not deal with any components (IDV and related) do not need this, however any time you create a pipeline or try to use any components in any way you would need to load and find them. This is at a pretty low level of the overall architecture.

While these might have been reasonable problems to solve for TLC, my gut feeling is that these goals are not in scope for ML.NET, except perhaps for the specific "zip" scenario mentioned in #600 (comment). No .NET Core application allows "drop in DLL replacement" deployment models and if we did, we would allow that kind of deployment update for a much broader range of applications.

ML.NET is a machine learning framework for .NET, not a dynamic loader or dependency resolver and/or DI framework.

I'd have different feedback about IDV: that is solving a schematization problem which definitely needs to be solved for a machine learning framework and which, to my knowledge, is not solved elsewhere in the .NET core ecosystem. Ideally the solution would be extracted and used more widely.

[Ivanidzo4ka]

Nothing brings more joy than throwing few principals under wheels of running train. @Zruty0 @TomFinley
I'm sure they have something to say :)

[dsyme]

Nothing brings more joy than throwing few principals under wheels of running train. @Zruty0 @TomFinley I'm sure they have something to say :)

Well, you're still only at version 0.3. The train just left the first station :)