subsetting a `Weights` vector does not return a weight vector

[floswald]

using DataFrames, DataFramesMeta, StatsBase

julia> d = DataFrame(a=1:10,b = 2.0 * (1:10),w=Weights([1,zeros(9)...]))
10×3 DataFrames.DataFrame
│ Row │ a  │ b    │ w   │
├─────┼────┼──────┼─────┤
│ 1   │ 1  │ 2.0  │ 1.0 │
│ 2   │ 2  │ 4.0  │ 0.0 │
│ 3   │ 3  │ 6.0  │ 0.0 │
│ 4   │ 4  │ 8.0  │ 0.0 │
│ 5   │ 5  │ 10.0 │ 0.0 │
│ 6   │ 6  │ 12.0 │ 0.0 │
│ 7   │ 7  │ 14.0 │ 0.0 │
│ 8   │ 8  │ 16.0 │ 0.0 │
│ 9   │ 9  │ 18.0 │ 0.0 │
│ 10  │ 10 │ 20.0 │ 0.0 │

julia> @linq d |>
       @where(:a .>3) |>
       @select(b=mean(:b,:w))
ERROR: ArgumentError: reduced dimension(s) must be integers
Stacktrace:
 [1] reduced_indices(::Tuple{Base.OneTo{Int64}}, ::Array{Float64,1}) at ./reducedim.jl:35
 [2] reducedim_initarray at ./reducedim.jl:73 [inlined]
 [3] mean(::Array{Float64,1}, ::Array{Float64,1}) at ./statistics.jl:57
 [4] (::###700#27{DataFrames.DataFrame})(::Array{Float64,1}, ::Array{Float64,1}) at /Users/florian.oswald/.julia/v0.6/DataFramesMeta/src/DataFramesMeta.jl:70

julia> @linq d |>
       @where(:a .>3) |>
       @select(b=StatsBase.mean(:b,:w))
ERROR: MethodError: Cannot `convert` an object of type Expr to an object of type DataFramesMeta.SymbolParameter
This may have arisen from a call to the constructor DataFramesMeta.SymbolParameter(...),
since type constructors fall back to convert methods.

[floswald]

just modifying

Base.getindex(wv::AbstractWeights, i) = Weights(getindex(wv.values, i))

doesn't do the trick. a whole raft of methods assumes that this returns an Array. starting with the print method

julia> using StatsBase

julia> v = Weights(rand(3))
3-element StatsBase.Weights{Float64,Float64,Array{Float64,1}}:
Error showing value of type StatsBase.Weights{Float64,Float64,Array{Float64,1}}:
ERROR: MethodError: Cannot `convert` an object of type Float64 to an object of type StatsBase.Weights
This may have arisen from a call to the constructor StatsBase.Weights(...),
since type constructors fall back to convert methods.
Stacktrace:
 [1] _getindex at ./abstractarray.jl:923 [inlined]
 [2] getindex(::StatsBase.Weights{Float64,Float64,Array{Float64,1}}, ::Int64, ::Int64) at ./abstractarray.jl:882
 [3] isassigned(::StatsBase.Weights{Float64,Float64,Array{Float64,1}}, ::Int64, ::Int64, ::Vararg{Int64,N} where N) at ./abstractarray.jl:222
 [4] alignment(::IOContext{Base.Terminals.TTYTerminal}, ::StatsBase.Weights{Float64,Float64,Array{Float64,1}}, ::Array{Int64,1}, ::Array{Int64,1}, ::Int64, ::Int64, ::Int64) at ./show.jl:1357
 [5] print_matrix(::IOContext{Base.Terminals.TTYTerminal}, ::StatsBase.Weights{Float64,Float64,Array{Float64,1}}, ::String, ::String, ::String, ::String, ::String, ::String, ::Int64, ::Int64) at ./show.jl:1487
 [6] print_matrix(::IOContext{Base.Terminals.TTYTerminal}, ::StatsBase.Weights{Float64,Float64,Array{Float64,1}}, ::String, ::String, ::String) at ./show.jl:1459
 [7] #showarray#263(::Bool, ::Function, ::IOContext{Base.Terminals.TTYTerminal}, ::StatsBase.Weights{Float64,Float64,Array{Float64,1}}, ::Bool) at ./show.jl:1709
 [8] display(::Base.REPL.REPLDisplay{Base.REPL.LineEditREPL}, ::MIME{Symbol("text/plain")}, ::StatsBase.Weights{Float64,Float64,Array{Float64,1}}) at ./REPL.jl:122
 [9] display(::Base.REPL.REPLDisplay{Base.REPL.LineEditREPL}, ::StatsBase.Weights{Float64,Float64,Array{Float64,1}}) at ./REPL.jl:125
 [10] display(::StatsBase.Weights{Float64,Float64,Array{Float64,1}}) at ./multimedia.jl:194
 [11] eval(::Module, ::Any) at ./boot.jl:235
 [12] print_response(::Base.Terminals.TTYTerminal, ::Any, ::Void, ::Bool, ::Bool, ::Void) at ./REPL.jl:144
 [13] print_response(::Base.REPL.LineEditREPL, ::Any, ::Void, ::Bool, ::Bool) at ./REPL.jl:129
 [14] (::Base.REPL.#do_respond#16{Bool,Base.REPL.##26#36{Base.REPL.LineEditREPL,Base.REPL.REPLHistoryProvider},Base.REPL.LineEditREPL,Base.LineEdit.Prompt})(::Base.LineEdit.MIState, ::Base.AbstractIOBuffer{Array{UInt8,1}}, ::Bool) at ./REPL.jl:646

any guidance on how to fix this?

[floswald]

also: what's in fact the right way to proceed here, i.e. what is the correct subset of a weights vector? in particular, should one carry over the sum of weights from the full vector into the subset or compute a new sum of weights (as I'm doing above)?

[nalimilan]

Thanks for investigating this. The problem isn't that getindex is supposed to return an array. On the contrary, it's that your new method is called even when i is an Int, so you're trying to wrap a single weight in a Weights vector. We should override getindex(::Weights, ::AbstractVector{<:Integer}) and getindex(::Weights, ::AbstractVector{Bool}) instead. Or maybe easier and more robust: pass i to getindex on the wrapped Array, check whether the result is an Array, and if so wrap it in a weights vector. See how CategoricalArrays does it:
https://github.com/JuliaData/CategoricalArrays.jl/blob/eb8556e5022c62168d660007387b4036acf381b1/src/array.jl#L534-L547

I don't think we need to do anything special for the sum of weights: it just needs to be recomputed for the new vector, which the constructor will do.

[nalimilan]

Fixed for a long time by #526.