Some concrete types are not properly interned

[Keno]

Internally in the system, we rely on the fact that concrete types are comparable by pointers. However, it is possible to create type signatures that are == to concrete types, but are not normalized to the corresponding type, causing all sorts of problems. Example:

julia> f(x::Int8, y::Int8) = 1
f (generic function with 1 method)

julia> f(x::T, y::T) where {T <: Int8} = 2
julia-debug: /home/keno/julia-comparison/src/gf.c:1105: check_ambiguous_visitor: Assertion `closure->after' failed.

[martinholters]

Ref. #25430 and #25796.

[martinholters]

Let me give a small recap of my thoughts on #25430, if I still remember correctly...

The issue are types like these:

julia> T1 = Tuple{Int8}
Tuple{Int8}

julia> T2 = Tuple{T} where T<:Int8
Tuple{T} where T<:Int8

julia> isconcretetype(T1)
true

julia> T1 === T2
false

julia> T1 == T2
true

As noted, we have several places where we assume that if isconcretetype(T1), then T1 == T2 can be simplified to T1 === T2, i.e. a much faster pointer comparison. Apparently, that does not work in the example above. We could make the cache lookup more aggressive (at the cost of additional runtime during type construction), but if Tuple{T} where T<:Int8 were to be constructed before the first Tuple{Int8}, we would cache the former and then later normalize the latter to the former, which we probably don't want. Hence, in #25796, I fixed subtyping/type intersection to be more careful about when to assume that T1 == T2 can be simplified to T1 === T2. Unfortunately, besides performance concerns, this lead to a problem with recursive type definitions which I couldn't resolve, hence it stalled. EDIT: misremembered, I actually could resolve it. Going forward, one option might be to revive #25796 and try to apply the same fixes to whatever code is the culprit for the method definition in this issue.

Alternatively, we could try to normalize types to equal concrete types (if possible) during construction, even if the respective concrete type has not been constructed/cached before. I'm doubtful, however, whether that can be achieved in general.

[martinholters]

Ah, #25796 almost fixes this, see #25796 (comment).

[chethega]

FWIW, this also manifests in

julia> T2[(1,)] .= T1[(1,)]
1-element Array{Tuple{T} where T<:Int8,1}:

signal (11): Segmentation fault

[raminammour]

Even uglier is this silent monstrosity 🤣 :

julia> T1[(1,)] .= T2[(1,)]
1-element Array{Tuple{Int8},1}:
 (-112,)

julia> T1[(1,)] .= T2[(1,)]
1-element Array{Tuple{Int8},1}:
 (112,)

julia> T1[(1,)] .= T2[(1,)]
1-element Array{Tuple{Int8},1}:
 (16,)

julia> versioninfo()
Julia Version 1.3.0-alpha.0
Commit 6c11e7c2c4 (2019-07-23 01:46 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  CPU: Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)
Environment:
  JULIA_EDITOR = /Applications/Sublime\ Text.app/Contents/MacOS/Sublime\ Text