Create a function to use type inference for eltype

[Tortar]

I'm using this function to compute the type generically for iterators:

function compute_eltype(iter)
    T = eltype(iter)
    return T === Any ? Base.@default_eltype(iter) : T
end

and I'm asking myself if this couldn't be applied by default to some generators and iterators which currently returns Any (actually if it is applied specifically to just those types it can be used only Base.@default_eltype(iter) I think)

For example consider

julia> iter = Iterators.flatten(((3,), [4, 5]))
Base.Iterators.Flatten{Tuple{Tuple{Int64}, Vector{Int64}}}(((3,), [4, 5]))

julia> iter = Iterators.flatten(((3,), [4, 5]))

julia> eltype(iter)
Any

julia> compute_eltype(iter)
Int64

julia> iter = (x^2 for x in 1:10)
Base.Generator{UnitRange{Int64}, var"#3#4"}(var"#3#4"(), 1:10)

julia> eltype(iter)
Any

julia> compute_eltype(iter)
Int64

Performance-wise I see a cost of a couple of ns. I found that there is this other issue and associated PR about a special case of this: #48249 and #48277, but I think that a function as above could be applied more generally. Would this be okay? Or is this problematic?

[nsajko]

As far as I remember, Base.@default_eltype relies on type inference. Type inference works on a best-effort basis, so it's good for optimizations in some cases, but mustn't be relied on for correctness. So the change you're proposing wouldn't be correct.

Apart from that, the run time cost of type inference wouldn't be acceptable for eltype.

[Tortar]

Consider that though collect(::Base.Generator) relies on Base.@default_eltype to infer the type for the array. So it would be logical in my opinion to use it for eltype.

Also I see very good performance using it on some micro-benchmarks:

julia> using BenchmarkTools

julia> iter = (x for x in 1:10)
Base.Generator{UnitRange{Int64}, typeof(identity)}(identity, 1:10)

julia> @benchmark eltype($iter)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  1.182 ns … 6.061 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.192 ns             ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.197 ns ± 0.056 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▂        █▆       ▄▃                 ▁         ▃▂       ▁ ▁
  █▁▁▁▁▁▁▁▁██▁▁▁▁▁▁▁██▁▁▁▁▁▁▁▁▄▁▁▁▁▁▁▁▁██▁▁▁▁▁▁▁▁██▁▁▁▁▁▁▁█ █
  1.18 ns     Histogram: log(frequency) by time     1.24 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

julia> @benchmark Base.@default_eltype($iter)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  1.182 ns … 6.843 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.192 ns             ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.202 ns ± 0.085 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

           █                                                 
  ▃▁▁▁▁▁▁▁▁█▆▁▁▁▁▁▁▁▂▂▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▃▂▁▁▁▁▁▁▁▄▃▁▁▁▁▁▁▁▁▃ ▂
  1.18 ns        Histogram: frequency by time       1.24 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

julia> iter = (x < 5 ? x : 2.0*x for x in 1:10)
Base.Generator{UnitRange{Int64}, var"#1#2"}(var"#1#2"(), 1:10)

julia> @benchmark eltype($iter)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  1.182 ns … 22.753 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.192 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.203 ns ±  0.218 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

           █                                                  
  ▃▁▁▁▁▁▁▁▁█▆▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▄▃▁▁▁▁▁▁▁▁▃ ▂
  1.18 ns        Histogram: frequency by time        1.24 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

julia> @benchmark Base.@default_eltype($iter)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  1.182 ns … 6.622 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.193 ns             ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.209 ns ± 0.073 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

           █                                                 
  ▂▁▁▁▁▁▁▁▁█▆▁▁▁▁▁▁▁▃▂▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▃▂▁▁▁▁▁▁▁▇▄▁▁▁▁▁▁▁▁▅ ▂
  1.18 ns        Histogram: frequency by time       1.24 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

[nsajko]

xref #54207

[Tortar]

Indeed, it would be really okay if @default_eltype was just public API so that I could just use it as a fallback with confidence when eltype returns Any which can be also mentioned in the docstring:

infer_eltype(itr) = (T = eltype(itr)) === Any ? @default_eltype(itr) : T

[Tortar]

Another possibility is to create a new function as above and export it or even something maybe a bit stronger as

infer_eltype(itr) = ifelse((T2 = Base.@default_eltype(itr)) <: (T1 = eltype(itr)), T2, T1)

with all the needed disclaimers in the docstring

[LilithHafner]

We might want to special case Union{} because sometimes type inference can infer that the itr is empty, in which case it tells us Union{} which is true and precise and not usually what folks are looking for

julia> Base.@default_eltype Iterators.Filter([1,2,3]) do x
           sum([1]) == 0
       end
Int64

julia> Base.@default_eltype Iterators.Filter([1,2,3]) do x
           sum(1) == 0
       end
Union{}

[Tortar]

A thing I don't understand is why in Base.@default_eltype (https://github.com/JuliaLang/julia/blob/master/base/array.jl#L744) it is used promote_typejoin_union(T), I would prefer to infer a Union of concrete types than an abstract type e.g.

julia> s = (x < 5 ? x : 2.0*x for x in 1:10)
Base.Generator{UnitRange{Int64}, var"#5#6"}(var"#5#6"(), 1:10)

julia> T = Core.Compiler.return_type(Base._iterator_upper_bound, Tuple{typeof(s)})
Union{Float64, Int64}

julia> Base.promote_typejoin_union(T)
Real

Why can't we use Core.Compiler.return_type(Base._iterator_upper_bound, Tuple{typeof(s)}) at least when the Union is not too big? Notice that this affects also collect:

julia> collect(s)
10-element Vector{Real}:
...

[vtjnash]

collect does not return a Union, as it gets the types from the real values and not from inference

[Tortar]

right, now I see, it widens the type. Nonetheless I guess with an infer_eltype function which returns also Unions one can probably take advantage of this with collect(infer_eltype(itr), itr)

[Tortar]

@vtjnash this is a bit derailing the conversation, but I'd like to ask you: is it a bad idea to return a Union (of not too many types)? e.g. I was able to return a Vector{Union{Int, Float64}} in the above case by changing https://github.com/JuliaLang/julia/blob/master/base/array.jl#L822 in new = similar(dest, Union{T, typeof(el)}).

[vtjnash]

Yes. That will never be the type of the object when it actually contains data, so it slows down the case of when there is data to benefit the case where there is not data

[vtjnash]

You typically want the static and dynamic types to nearly coincide, for optimal results

[aplavin]

We might want to special case Union{} because sometimes type inference can infer that the itr is empty, in which case it tells us Union{} which is true and precise and not usually what folks are looking for

It would be really unfortunate, especially if this avoidance of Union{} is motivated solely by current compiler limitations (and not language design concerns).

Do you suggest turning Union{} into Any, or something else? There's just no other type that would retain the nice invariant that the (inferred) eltype of concatenation of A + B is a supertype of A and B eltypes.

[Tortar]

If we are going the route where we export a new function, which to me seems preferable, I (maybe mis)understood what @LilithHafner has suggested to be implemented as for example

function infer_eltype(itr)
    T1, T2 = eltype(itr), Base.@default_eltype(itr)
    ifelse(T2 !== Union{} && T2 <: T1, T2, T1)
end