Choice Prompt not behaving properly in Bot Framework V4

[Monomoy88]

Version

Microsoft.Bot.Builder.Azure, Version=4.5.2.0
Microsoft.Bot.Builder.Dialogs, Version=4.5.2.0

I've updated the version to 4.10.3 but problem still exists.

Describe the bug

I've created a Dialog bot which is responsible for showing ChoicePrompts based on QnAMaker contextual question answers.

var promptOptions = new PromptOptions
{
    Prompt = MessageFactory.Text(botResponse),
    Style = ListStyle.SuggestedAction,
    Choices = promptChoices
};

return await stepContext.PromptAsync(nameof(ChoicePrompt), promptOptions, cancellationToken);

As I'm using choice prompt so in the next waterfall step I'm trying to read the user choice using below code
var response= ((FoundChoice)stepContext.Result).Value;

In a certain case I've two options like below

While I'm choosing the 2nd option I'm getting 1st value in the stepContext.Result however stepContext.Context.Activity.Text is showing proper value that I've selected from bot.

Most of the cases its working fine i.e. value of stepContext.Result and stepContext.Context.Activity.Text are same.
Is it happening for the special characters >= and < ?
If so then what is the workaround regarding this?

Expected behavior

((FoundChoice)stepContext.Result).Value & stepContext.Context.Activity.Text should return same value if ChoicePrompt is used to render prompt in previous waterfall step of a dialog bot.

[v-kydela]

@Monomoy88 - I've tracked this bug down to Find.FindValues. Apparently the code tokenizes the utterance and each of the choices by getting rid of all special characters, so your two choices look identical to the recognizer. Because the two choices are identical to each other and to the utterance, they'll both get a perfect score. The tie is resolved by picking the longer string because the choices get sorted by length in the beginning. You can reproduce this behavior in a console app like this:

var choices = new List<string>
{
    "!@#$foo bar",
    "foo  bar%^&*",
};

// Pick the shorter one as the user utterance
var utterance = choices[0];
var opt = new FindValuesOptions();
var results = Find.FindValues(
    utterance,
    choices.Select((value, index) => new SortedValue { Value = value, Index = index }).ToList(),
    opt);

// The shorter choice was used:
Console.WriteLine(utterance);
// !@#$foo bar

// The text has been reduced to just the alphabet characters
// and the longer choice has been matched:
Console.WriteLine(JToken.FromObject(results));
// [
//   {
//     "Text": "foo bar",
//     "Start": 4,
//     "End": 10,
//     "TypeName": "value",
//     "Resolution": {
//       "value": "foo  bar%^&*",
//       "index": 1,
//       "score": 1.0
//     }
//   }
// ]

You have a few options for how to resolve this on your end:

1. Name the choices differently in your knowledge base so that distinct choices always contain distinct word characters (you could for example say "less than" instead of "<")
2. Create the suggested actions in your prompt explicitly and make them send the choice as an index ("0" or "1") instead of the full choice text
3. Create your own tokenizer function for your prompt's RecognizerOptions property so that it can recognize special characters as tokens

I will leave this open as a bug so the team can triage it properly. I'm considering it a bug because exact matches should always be preferred, and right now what's preferred is the longest choice with the highest score that disregards non-word characters.

[Monomoy88]

Thanks for explaining the fact in details.
Our solution is very generic & client facing where we can't restrict user to put special characters as long it is supported by QnA Maker.
In QnA Maker if it can distinguish between these two option then recognizer should do that too.
As of now I guess the 3rd option is more appropriate for our solution however I think the approach you proposed in 3rd option should be it's default behavior.

[mrivera-ms]

@v-kydela , are you able to take this bug? Please let us know if you have the availability.