Skip to content

BaseLine Mismatching for various tests when run against Release-Intrinsics #1096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
25 of 30 tasks
Anipik opened this issue Sep 28, 2018 · 5 comments
Closed
25 of 30 tasks

BaseLine Mismatching for various tests when run against Release-Intrinsics #1096

Anipik opened this issue Sep 28, 2018 · 5 comments
Assignees
@frank-dong-ms-zz
Labels
enhancement New feature or request P1 Priority of the issue for triage purpose: Needs to be fixed soon. test related to tests

Comments

@Anipik
Copy link
Contributor

Anipik commented Sep 28, 2018 •
edited
Loading

The List of the tests that fail are

  • TestCrossValidationMacro
  • BinaryClassifierLogisticRegressionBinNormTest
  • BinaryClassifierLogisticRegressionGaussianNormTest
  • BinaryClassifierLogisticRegressionNonNegativeTest
  • BinaryClassifierLogisticRegressionNormTest
  • BinaryClassifierTesterThresholdingTest
  • BinaryClassifierSymSgdTest
  • BinaryClassifierLogisticRegressionTest
  • BinaryClassifierPerceptronTest
  • MulticlassLRNonNegativeTest
  • DefaultCalibratorPerceptronTest
  • PAVCalibratorPerceptronTest
  • FastTreeBinaryClassificationCategoricalSplitTest
  • LinearClassifierTest
  • NoCalibratorLinearSvmTest
  • PAVCalibratorLinearSvmTest
  • PcaAnomalyTest
  • RandomCalibratorPerceptronTest
  • RegressorOlsTestOne
  • LpGcNormAndWhiteningWorkout
  • MultiClassificationLRSaveModelToOnnxTest
  • BinaryClassificationLRSaveModelToOnnxTest
  • EntryPointLinearPredictorSummary
  • EntryPointPcaPredictorSummary
  • BinaryClassifierFieldAwareFactorizationMachineTest
  • PcaWorkout
  • RffWorkout
  • SavePipeSsaSpikeNoData // unhandled exception
  • ChangePointDetectionWithSeasonality
  • EntryPointPipelineEnsemble
    All of these tests fail because the baseline numbers are different on netcoreapp3.0

cc @danmosemsft @eerhardt @shauheen @Ivanidzo4ka
@Anipik Anipik mentioned this issue Oct 1, 2018
Merged
@Anipik Anipik mentioned this issue Oct 19, 2018
Merged
@Anipik
Copy link
Contributor Author

Anipik commented Oct 19, 2018

@shauheen can you assign some one to look into following failures
RffWorkout
SavePipeSsaSpikeNoData // unhandled exception
ChangePointDetectionWithSeasonality
EntryPointPipelineEnsemble

These are not caused by mismatching in baseline files

@artidoro artidoro added the enhancement New feature or request label Oct 27, 2018
@Anipik Anipik mentioned this issue Nov 16, 2018
Merged
@Anipik
Copy link
Contributor Author

Anipik commented Nov 22, 2018 •
edited
Loading

Tests Disabled For Netcoreapp3.0

  • TestCrossValidationMacro
  • MulticlassLRTest
  • MulticlassLRNonNegativeTest
  • BinaryClassifierLogisticRegressionNonNegativeTest
  • BinaryClassifierLogisticRegressionBinNormTest
  • DefaultCalibratorPerceptronTest
  • PAVCalibratorPerceptronTest
  • RandomCalibratorPerceptronTest
  • MatrixFactorization
  • CommandTrainMlrWithStats
  • KmeansTest
  • GcnWorkout
  • RffWorkout
  • BinaryClassifierLogisticRegressionGaussianNormTest
  • SavePipeMovingAverageUniform
  • SavePipeSsaSpikeNoData
  • ChangePointDetectionWithSeasonality
  • TestSsaChangePointEstimator
  • TestSsaSpikeEstimator

@shmoradims shmoradims added the test related to tests label Nov 26, 2018
@Anipik Anipik mentioned this issue Nov 27, 2018
Merged
@glebuk
Copy link
Contributor

glebuk commented Jan 14, 2019

@Anipik - please investigate and summarize root causes

@antoniovs1029 antoniovs1029 added the P3 Doc bugs, questions, minor issues, etc. label Jan 10, 2020
@Anipik Anipik removed their assignment Feb 19, 2020
@frank-dong-ms-zz frank-dong-ms-zz added P1 Priority of the issue for triage purpose: Needs to be fixed soon. and removed P3 Doc bugs, questions, minor issues, etc. labels Apr 17, 2020
@frank-dong-ms-zz frank-dong-ms-zz self-assigned this Apr 17, 2020
@frank-dong-ms-zz
Copy link
Contributor

The difference comes from CPUMath using different instruction set:

  1. net framework and net core 2.1 uses CpuMathUtils.netstandard that uses SSE instruction set;
  2. net core 3.1 uses CpuMathUtils.netcoreapp that uses AVX, SSE or direct floating point calculation depending on hardward avaibility.
    AVX and SSE generates slightly different result due to nature of floating point math.
    Use below issue to track: support AVX instruction set at CPUMath native #5044

@frank-dong-ms-zz frank-dong-ms-zz mentioned this issue Apr 22, 2020
Merged
@frank-dong-ms-zz
Copy link
Contributor

created separate baseline for netcore app 31

@ghost ghost locked as resolved and limited conversation to collaborators Mar 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@frank-dong-ms-zz frank-dong-ms-zz

Labels
enhancement New feature or request P1 Priority of the issue for triage purpose: Needs to be fixed soon. test related to tests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@Anipik @artidoro @shmoradims @glebuk @antoniovs1029 @frank-dong-ms-zz