-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
There are a few tests that are disabled because the baseline comparisons fail on the 5th decimal for some of the numbers generated, on some OS.
As an example, the RegressorOlsTest() in PredictorTests fails just on the Mac debug version, because only one out of the generated 4896 predictions doesn't match:
baseline:
2625 5 5.09176636 0.091766357421875 0.0084210643544793129
Mac debug run predictions:
2625 5 5.091751 0.0917510986328125 0.0084182641003280878
I think we should make the tests more lenient to failures like this, modifying the comparison with the baseline to:
1- Have a sensitivity threeshold. Compare up to the 4th, or 6th decimal digit.
2- Count the failures, and declare the test as failed if 3% o the predictions/lines differ?
@justinormont @TomFinley @Zruty0 @zeahmed are those acceptable ranges?