Defining features using MicrosoftSchema in AzureAppConfiguration and appsetting.json does not work when not using variants

[emilssonn]

When AzureAppConfiguration loads features/config from Azure it checks for the existence of Variants, Allocation or Telemetry to decide what feature flag schema to use. If none of these are found it loads the config/feature in the DotnetSchema instead of the MicrosoftSchema. This makes merging features defined using the MicrosoftSchema in other config sources and the MicrosoftSchema impossible if not using a variant flag.

When following https://learn.microsoft.com/en-us/azure/azure-app-configuration/feature-management-dotnet-reference#feature-flag-declaration and then adding AzureAppConfiguration the feature flags defined in appsettings.json using MicrosoftSchema cannot be overridden using AzureAppConfiguration if not using variants.

Steps to reproduce:

1. Add feature flags in appsettings.json using the MicrosoftSchema and not using variants.
2. Connect to AzureAppConfiguration.
3. Add the same feature flag (not variant)
4. In AzureAppConfiguration, enabled/disable the feature flag.
5. Nothing changes in the application (restart to reload if needed)

There should be some way (or the library should have more logic) to force the library to always call ProcessMicrosoftSchemaFeatureFlag in this section:

AppConfiguration-DotnetProvider/src/Microsoft.Extensions.Configuration.AzureAppConfiguration/FeatureManagement/FeatureManagementKeyValueAdapter.cs

Line 34 in 7dc547c

if ((featureFlag.Variants != null && featureFlag.Variants.Any()) || featureFlag.Allocation != null || featureFlag.Telemetry != null)

[zhiyuanliang-ms]

Hi, @emilssonn Thanks for bringing up this issue.

AppConfiguration-DotnetProvider/src/Microsoft.Extensions.Configuration.AzureAppConfiguration/FeatureManagement/FeatureManagementKeyValueAdapter.cs

Line 34 in 7dc547c

if ((featureFlag.Variants != null && featureFlag.Variants.Any()) || featureFlag.Allocation != null || featureFlag.Telemetry != null)

You are right. The root cause of this bug is that if there is no variant or telemetry related fields in the feature flag, the flag will be converted into .NET schema.

And the feature management library will prioritize using Microsoft schema code

---

There are historical reasons why we still convert all non-variant feature flags to the .NET schema. In short, we want to maintain compatibility with both the classic .NET feature flag schema and the newer Microsoft schema.

Before releasing provider v8 and feature management v4, we did consider fully deprecating the .NET feature flag schema. However, in order to avoid breaking changes, we eventually decided to let the feature management library support both schemas. And we no longer add new functionality, such as variants and telemetry, to the older .NET schema.

We have observed that users tend to update our .NET provider (this library) to the latest version more frequently than they update the .NET feature management library (you can see this from NuGet download counts across different versions). We were concerned about a potential breaking change if someone used the latest version of the .NET provider but was still on a feature management library version earlier than 3.2.0 (which does not support the Microsoft feature flag schema at all). In that scenario, all feature flags would suddenly fail. To prevent this from happening, we continue converting non-variant feature flags to the older .NET feature flag schema.

---

To workaround with this, could you please try to write feature flag using .NET feature flag schema in your local appsettings.json file?

[emilssonn]

Using the DotnetSchema in local/appsettings.json config works. But this also means that it is not possible to have both none variant and variant feature flags at the same time, if you declare them in local config. So the example in https://learn.microsoft.com/en-us/azure/azure-app-configuration/feature-management-dotnet-reference#feature-flag-declaration is not possible when using Azure App Configuration aswell.
Shouldn't a setting in this library to always parse feature flags as MicrosoftSchema solve the issue?

[zhiyuanliang-ms]

@emilssonn

Shouldn't a setting in this library to always parse feature flags as MicrosoftSchema solve the issue?

This could be a solution! @zhenlan @jimmyca15 @rossgrambo @SamSadfa @amerjusupovic

[zhenlan]

I remember we discussed having a compatibility switch to force the Microsoft Schema if a customer chooses to. If we didn't add one, we should, probably as an environment variable (so it won't be persisted in the source code).

[emilssonn]

I looked some more into the ProcessMicrosoftSchemaFeatureFlag method and I think there is another issue that will make it hard to work with feature flags in Azure App Config and other config sources (appsettings.json) at the same time. Since the MicrosoftSchema is defined as an array of feature flags, the logic in ProcessMicrosoftSchemaFeatureFlag just uses a new index to set values? Maybe I misunderstand how the merging logic of arrays works in IConfiguration, but doesn't that mean it might update/set the wrong feature flag?

A setting to force MicrosoftSchema would still be good, just to be able to have one format of feature flags from Azure App Configuration.

[jimmyca15]

I am thinking that we can start at a high offset when defining the array to prevent flags defined by appsettings.json from colliding with flags defined in App Configuration. However, won't this won't fix is allowing appsetings.json to override the flags defined in App Configuration. I still think it's worth pursuing.

The start index can be static and incremented to support multiple app configuration providers in process if desired.

[jimmyca15]

I remember we discussed having a compatibility switch to force the Microsoft Schema if a customer chooses to. If we didn't add one, we should, probably as an environment variable (so it won't be persisted in the source code).

We will be adding an environment variable to force the use of the Microsoft Schema (newest schema).

Candidate name: AZURE_APP_CONFIGURATION_FM_SCHEMA_COMPATIBILITY_DISABLED

Accepted values: True | False (default)

@amerjusupovic