[Facebook and Webex adapter functional test] Validate required variables to correct inconclusive results #4585
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #4545
Description
The inconclusive result of Facebook's and Webex's tests are a product of required environment variables being un-reachable, which skips the actual functional tests. Our tests ran successfully whenever all environment variables were set up correctly, and we could only replicate the issue's reported behavior when leaving variables null purposefully.
It should be noted that the current inconclusive response is not an issue of the test's implementation. Changing the inconclusive assertions to failures would lead to false negatives, since the tests did not fail, but were unable to be run.
Our proposed solution is to integrate a required variable validating task to run prior to all other tasks in the pipeline. This keeps the tests intact and saves waiting for the pipeline to run until failure due to missing, empty or white-spaced variables, making those occurrences easier to catch and correct.
We validate for required variables mannually on each case, leaving room for custom behavior. It should be noted that we also left the FacebookTestBotFacebookVerifyToken variable on botbuilder-dotnet-ci-facebook-test.yml unvalidated as it was detected that it was not required for the test to run.
Specific Changes
Testing
The following image showcases the new task in action. First we deleted required variables, then the task outputs which variables were missing.
We also checked that both Facebook's and Webex's tests were running correctly, which we can corroborate with the following images.
Facebook:

Webex:
