diff between arrays of Int32 or Float32 shows "nothing"

[dpinol]

Int64 or Float64 array comparison correctly displays the differences

@testset ExtendedTestSet "ts" begin
       @test [Int64(2)]==[Int64(3)]
       end
....
Diff:
[2, 3]

But for Int32 or Float32 it shows "Nothing"

@testset ExtendedTestSet "ts" begin
       @test [Int32(2)]==[Int32(3)]
 end
....
Diff:
nothing

adding logs to this package I saw that for 32bits, the expressions that gets into Test.record is strangely a Ref

Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: Symbol ==
    2: Expr
      head: Symbol ref
      args: Array{Any}((2,))
        1: Symbol Int32
        2: Int64 2
    3: Expr
      head: Symbol ref
      args: Array{Any}((2,))
        1: Symbol Int32
        2: Int64 3

instead of a vector (which is what I get with 64bits)

Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: Symbol ==
    2: Expr
      head: Symbol vect
      args: Array{Any}((1,))
        1: Float64 2.0
    3: Expr
      head: Symbol vect
      args: Array{Any}((1,))
        1: Float64 3.0

[Teo-ShaoWei]

I would like to gain support to get a general solution like #17 approved.

Firstly regarding the analysis above, for the sake of anyone who also did dump(:([Int32(2)] == [Int32(3)])) and wonder where the :ref is. What happened when we evaluate @test [Int32(2)] == [Int32{3}] is that the failure data become the string "Int32[2] == Int32[3]". This is because Julia had evaluated the original expression.

Then this line:

TestSetExtensions.jl/src/TestSetExtensions.jl

Line 70 in 3fe472d

Meta.parse(res.data)

will create the expression :(Int32[2]==Int32[3])) to deepdiff on. Finally, taking the dump give us the :ref printout that @dpinol shared:

julia> dump(:(Int32[2]==Int32[3]))
Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: Symbol ==
    2: Expr
      head: Symbol ref
      args: Array{Any}((2,))
        1: Symbol Int32
        2: Int64 2
    3: Expr
      head: Symbol ref
      args: Array{Any}((2,))
        1: Symbol Int32
        2: Int64 3

The issue is once again due to the regression of #10, and we should be thinking of generalizing it. I think the only way to fix the issue is to return to just comparing the evaluated args field, and then beef up DeepDiff.jl to handle the possibilities.

[ReubenJ]

Just ran into this again when trying to diff Vectors not of type Int, String, etc.

I think the only way to fix the issue is to return to just comparing the evaluated args field

This is not possible in the cases where the comparison args hold any non-builtin types, because the eval won't have access to them in its evaluation context.

Unfortunately, at the level that this package works (extending Test.record with a new method), there is nothing that it can do beyond diffing the syntactic representation of the arguments in the comparison. In the new Test.record method we have access to a ExtendedTestSet and a Fail as arguments. The only reference to the original comparison that we have access to is the orig_expr and data fields of the Fail argument—both string representations of the original comparison arguments.

If the string representation is enough to get back to the actual value (in the case of Int, String, and the rest of the types listed here, then its sufficient to Meta.parse/eval the string representation and deepdiff the result. This is the solution suggested in #17. However, if we can't eval the output of Meta.parse (like for MyPackage.MyType(1) == MyPackage.MyType(2)), then there's nothing more that can be done from within the Test.record method. I'm pretty sure the only way to get proper diffing here would be to create a special @test_diff macro, or make some changes to the @test macro in Base.

Apologies for bringing this out of the shadows after 4 years, but I just spent a bunch of time playing around with ways to solve the issue, and wanted to make sure my efforts were recorded somewhere in case someone else wants to pick this up.