OpenBLAS generates random result in MXNet forward

[hetong007]

Recently in MXNet there's an issue about random model output when given fixed model and input, if computation happens on CPU with OpenBLAS: apache/mxnet#11853

We found it is reproducible with OpenBLAS 0.3.1, but not reproducible with 0.2.20. It is easier to reproduce on AWS p3.16xlarge instance with Intel(R) Xeon(R) CPU E5-2686 v4.

I don't have enough knowledge to track down to the breaking point. Instead I searched a bit and now guess it may be related with #1666 and JuliaLang/LinearAlgebra.jl#539.

[brada4]

If you read julia discussion in your own link it is not reproducible with 0.3.0 and devel, i.e can you confirm that only 0.3.1 is broken?

[martin-frbg]

0.3.2 should be released this weekend. You could try the current "develop" branch (or just apply the changes from PR JuliaLang/julia#1667 to 0.3.1 - you can get them in patch format by appending ".diff" to the PR number in the github interface) to see if the fix from JuliaLang/julia#1666 actually solves your problem.

[hetong007]

@martin-frbg Thanks for the information. I tested it and seems 0.3.2.dev is good.

[szha]

@martin-frbg looking forward to the release

[martin-frbg]

Closing as 0.3.2 is released now.