-
Notifications
You must be signed in to change notification settings - Fork 13
Suppor dimensional metadata #62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
5ae5ab0
47daf4c
e1b08c5
19287e0
10ca2f0
bbe4fc7
ffbb582
3a0466f
1a418e3
115962f
7cb0aef
0d5bb5c
4a11c37
8225a7e
2ae2768
57e4f8c
e68ece2
c35f7a1
a3ecbcd
b519d26
ab256ec
838047a
c0ae557
2e343f4
b01d2c2
6a0af43
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
name = "DataAPI" | ||
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a" | ||
authors = ["quinnj <[email protected]>"] | ||
version = "1.15.0" | ||
version = "1.16.0" | ||
|
||
[compat] | ||
julia = "1" | ||
|
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
|
@@ -322,20 +322,6 @@ The `write` field indicates whether modifying metadata with the [`metadata!`](@r | |||||||
""" | ||||||||
metadatasupport(::Type) = (read=false, write=false) | ||||||||
|
||||||||
""" | ||||||||
colmetadatasupport(T::Type) | ||||||||
|
||||||||
Return a `NamedTuple{(:read, :write), Tuple{Bool, Bool}}` indicating whether | ||||||||
values of type `T` support column metadata. | ||||||||
|
||||||||
The `read` field indicates whether reading metadata with the [`colmetadata`](@ref) | ||||||||
and [`colmetadatakeys`](@ref) functions is supported. | ||||||||
|
||||||||
The `write` field indicates whether modifying metadata with the [`colmetadata!`](@ref), | ||||||||
[`deletecolmetadata!`](@ref), and [`emptycolmetadata!`](@ref) functions is supported. | ||||||||
""" | ||||||||
colmetadatasupport(::Type) = (read=false, write=false) | ||||||||
|
||||||||
""" | ||||||||
metadata(x, key::AbstractString, [default]; style::Bool=false) | ||||||||
|
||||||||
|
@@ -381,6 +367,9 @@ end | |||||||
Return an iterator of metadata keys for which `metadata(x, key)` returns a | ||||||||
metadata value. | ||||||||
Throw an error if `x` does not support reading metadata. | ||||||||
|
||||||||
If `metadatasupport(typeof(x)).read` or `metadatasupport(typeof(x)).write` return `true` | ||||||||
this should also be defined. | ||||||||
""" | ||||||||
function metadatakeys end | ||||||||
|
||||||||
|
@@ -412,6 +401,20 @@ Throw an error if `x` does not support metadata deletion. | |||||||
""" | ||||||||
function emptymetadata! end | ||||||||
|
||||||||
""" | ||||||||
colmetadatasupport(T::Type) | ||||||||
|
||||||||
Return a `NamedTuple{(:read, :write), Tuple{Bool, Bool}}` indicating whether | ||||||||
values of type `T` support column metadata. | ||||||||
|
||||||||
The `read` field indicates whether reading metadata with the [`colmetadata`](@ref) | ||||||||
and [`colmetadatakeys`](@ref) functions is supported. | ||||||||
|
||||||||
The `write` field indicates whether modifying metadata with the [`colmetadata!`](@ref), | ||||||||
[`deletecolmetadata!`](@ref), and [`emptycolmetadata!`](@ref) functions is supported. | ||||||||
""" | ||||||||
colmetadatasupport(::Type) = (read=false, write=false) | ||||||||
|
||||||||
""" | ||||||||
colmetadata(x, col, key::AbstractString, [default]; style::Bool=false) | ||||||||
|
||||||||
|
@@ -472,7 +475,7 @@ end | |||||||
colmetadatakeys(x, [col]) | ||||||||
|
||||||||
If `col` is passed return an iterator of metadata keys for which | ||||||||
`metadata(x, col, key)` returns a metadata value. Throw an error if `x` does not | ||||||||
`colmetadata(x, col, key)` returns a metadata value. Throw an error if `x` does not | ||||||||
support reading column metadata or if `col` is not a column of `x`. | ||||||||
|
||||||||
`col` must have a type that is supported by table `x` for column indexing. | ||||||||
|
@@ -481,6 +484,9 @@ Following the Tables.jl contract `Symbol` and `Int` are always allowed. | |||||||
If `col` is not passed return an iterator of `col => colmetadatakeys(x, col)` | ||||||||
pairs for all columns that have metadata, where `col` are `Symbol`. | ||||||||
If `x` does not support column metadata return `()`. | ||||||||
|
||||||||
This method must be defined if `colmetadatasupport(typeof(x)).read` or | ||||||||
`colmetadatasupport(typeof(x)).write` return `true`. | ||||||||
""" | ||||||||
function colmetadatakeys end | ||||||||
|
||||||||
|
@@ -515,6 +521,116 @@ Throw an error if `x` does not support metadata deletion for column `col`. | |||||||
""" | ||||||||
function emptycolmetadata! end | ||||||||
|
||||||||
""" | ||||||||
dimmetadatasupport(T::Type, dim::Int) | ||||||||
|
||||||||
Return a `NamedTuple{(:read, :write), Tuple{Bool, Bool}}` indicating whether | ||||||||
values of type `T` support metadata correspond to dimension `dim`. | ||||||||
Comment on lines
+525
to
+528
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry, I missed this substantial point: There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this is could be OK as is proposed here. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What do you mean by "there is exactly one dimension always"? :-) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK I get it now, you mean that columns are the second dimension so I still think there's a problem though: |
||||||||
|
||||||||
The `read` field indicates whether reading metadata with the [`dimmetadata`](@ref) | ||||||||
and [`dimmetadatakeys`]](@ref) functions is supported. | ||||||||
|
||||||||
The `write` field indicates whether modifying metadata with the [`dimmetadata!`](@ref), | ||||||||
[`deletemetadata!`](@ref), and [`emptymetadata!`](@ref) functions is supported. | ||||||||
""" | ||||||||
dimmetadatasupport(::Type, i::Int) = (read=false, write=false) | ||||||||
Tokazama marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
|
||||||||
""" | ||||||||
dimmetadata(x, dim::Int, key::AbstractString, [default]; style::Bool=false) | ||||||||
|
||||||||
Return metadata value associated with object `x` for dimension `dim` and key `key`. | ||||||||
Throw an error if `x` does not support reading metadata for dimension `dim` or `x` | ||||||||
supports reading metadata, but does not have a mapping for dimension `dim` for `key`. | ||||||||
|
||||||||
If `style=true` return a tuple of metadata value and metadata style. Metadata | ||||||||
style is an additional information about the kind of metadata that is stored for | ||||||||
the `key`. | ||||||||
|
||||||||
$STYLE_INFO | ||||||||
|
||||||||
If `default` is passed then return it if `x` supports reading metadata and has | ||||||||
dimension `dim` but mapping for `key` is missing. | ||||||||
If `style=true` return `(default, :default)`. | ||||||||
""" | ||||||||
function dimmetadata end | ||||||||
|
||||||||
""" | ||||||||
dimmetadata(x, [dim::Int]; style::Bool=false) | ||||||||
|
||||||||
If `dim` is not passed, return a dictionary mapping dimensions that have | ||||||||
associated metadata to dictionaries mapping all metadata keys to metadata | ||||||||
values associated with object `x` for a given dimension, so that | ||||||||
`colmetadata(x)[dim][key] == dimmetadata(x, dim, key)`. | ||||||||
|
||||||||
If `dim` is passed return a dictionary mapping all column metadata keys | ||||||||
to metadata values associated with dimension `dim` of object `x`, so that | ||||||||
`colmetadata(x, dim)[key] == dimmetadata(x, dim, key)`. | ||||||||
Throw an error if `x` does not support reading metadata for dimension `dim` | ||||||||
or `dim` is not dimension of `x`. | ||||||||
|
||||||||
If `style=true` values are tuples of metadata value and metadata style. Metadata | ||||||||
style is an additional information about the kind of metadata that is stored for | ||||||||
the `key`. | ||||||||
|
||||||||
$STYLE_INFO | ||||||||
|
||||||||
The returned object may be freshly allocated on each call to `dimmetadata` and | ||||||||
is considered to be owned by `x` so it must not be modified. | ||||||||
""" | ||||||||
function dimmetadata(x::T, dim::Int; style::Bool=false) where {T} | ||||||||
if !dimmetadatasupport(T, dim).read | ||||||||
throw(ArgumentError("Objects of type $T do not support reading dimension metadata")) | ||||||||
end | ||||||||
return Dict(key => dimmetadata(x, dim, key, style=style) for key in dimmetadatakeys(x, dim)) | ||||||||
end | ||||||||
function dimmetadata(x; style::Bool=false) | ||||||||
Tuple(dimmetadata(x, dim; style=style) for dim in 1:ndims(x)) | ||||||||
end | ||||||||
|
||||||||
""" | ||||||||
dimmetadatakeys(x, [dim::Int]) | ||||||||
|
||||||||
If `dim` is passed return an iterator of metadata keys for which | ||||||||
`dimmetadata(x, dim, key)` returns a metadata value. Throw an error if `x` does not | ||||||||
support reading dimension metadata or if `dim` is not a dimension of `x`. | ||||||||
|
||||||||
If `dim` is not passed return an iterator of `dim => dimmetadatakeys(x, dim)` | ||||||||
pairs for all dimensions that have metadata. | ||||||||
|
||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's do the same as for
Suggested change
|
||||||||
This method must be defined if `dimmetadatasupport(typeof(x)).read` or | ||||||||
`dimmetadatasupport(typeof(x)).write` return `true`. | ||||||||
""" | ||||||||
function dimmetadatakeys end | ||||||||
|
||||||||
""" | ||||||||
dimmetadata!(x, dim::Int, key::AbstractString, value; style::Symbol=:default) | ||||||||
|
||||||||
Set metadata for object `x` for dimension `dim` for key `key` to have value `value` | ||||||||
and style `style` (`:default` by default) and return `x`. | ||||||||
Throw an error if `x` does not support setting metadata for dimension `dim`. | ||||||||
|
||||||||
$STYLE_INFO | ||||||||
""" | ||||||||
function dimmetadata! end | ||||||||
|
||||||||
""" | ||||||||
deletedimmetadata!(x, dim::Int, key::AbstractString) | ||||||||
|
||||||||
Delete metadata for object `x` for dimension `dim` for key `key` and return `x` | ||||||||
(if metadata for `key` is not present do not perform any action). | ||||||||
Throw an error if `x` does not support metadata deletion for dimension `dim`. | ||||||||
""" | ||||||||
function deletedimmetadata! end | ||||||||
|
||||||||
""" | ||||||||
emptydimmetadata!(x, dim::Int) | ||||||||
|
||||||||
Delete all metadata for object `x` for to dimension `dim` and return `x`. | ||||||||
If `dim` is not passed delete all dimension level metadata for table `x`. | ||||||||
Throw an error if `x` does not support metadata deletion for dimension `dim`. | ||||||||
Tokazama marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
""" | ||||||||
function emptydimmetadata! end | ||||||||
|
||||||||
""" | ||||||||
rownumber(row) | ||||||||
|
||||||||
|
Original file line number | Diff line number | Diff line change | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -17,12 +17,23 @@ DataAPI.levels(x::TestArray) = reverse(DataAPI.levels(x.x)) | |||||||||||||
struct TestMeta | ||||||||||||||
table::Dict{String, Any} | ||||||||||||||
col::Dict{Symbol, Dict{String, Any}} | ||||||||||||||
dims::Tuple{Dict{String, Any}, Dict{String, Any}} | ||||||||||||||
|
||||||||||||||
TestMeta() = new(Dict{String, Any}(), Dict{Symbol, Dict{String, Any}}()) | ||||||||||||||
function TestMeta() | ||||||||||||||
new(Dict{String, Any}(), Dict{Symbol, Dict{String, Any}}(), (Dict{String, Any}(), Dict{String, Any}())) | ||||||||||||||
end | ||||||||||||||
end | ||||||||||||||
Base.ndims(::TestMeta) = 2 | ||||||||||||||
|
||||||||||||||
DataAPI.metadatasupport(::Type{TestMeta}) = (read=true, write=true) | ||||||||||||||
DataAPI.colmetadatasupport(::Type{TestMeta}) = (read=true, write=true) | ||||||||||||||
function DataAPI.dimmetadatasupport(::Type{TestMeta}, dim::Int) | ||||||||||||||
if dim == 1 || dim == 2 | ||||||||||||||
(read=true, write=true) | ||||||||||||||
else | ||||||||||||||
(read=false, write=false) | ||||||||||||||
end | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.metadata(x::TestMeta, key::AbstractString; style::Bool=false) | ||||||||||||||
return style ? x.table[key] : x.table[key][1] | ||||||||||||||
|
@@ -93,6 +104,39 @@ end | |||||||||||||
|
||||||||||||||
DataAPI.emptycolmetadata!(x::TestMeta) = empty!(x.col) | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadata(x::TestMeta, dim::Int, key::AbstractString; style::Bool=false) | ||||||||||||||
return style ? x.dims[dim][key] : x.dims[dim][key][1] | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadata(x::TestMeta, dim::Int, key::AbstractString, default; style::Bool=false) | ||||||||||||||
haskey(x.dims[dim], key) && return DataAPI.metadata(x, dim, key, style=style) | ||||||||||||||
return style ? (default, :default) : default | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadatakeys(x::TestMeta, dim::Int) | ||||||||||||||
if DataAPI.dimmetadatasupport(TestMeta, dim).read | ||||||||||||||
return keys(x.dims[dim]) | ||||||||||||||
else | ||||||||||||||
return () | ||||||||||||||
end | ||||||||||||||
Comment on lines
+117
to
+121
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The docstring says we should throw an error when
Suggested change
|
||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.dimmetadata!(x::TestMeta, dim::Int, key::AbstractString, value; style::Symbol=:default) | ||||||||||||||
x.dims[dim][key] = (value, style) | ||||||||||||||
return x | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.deletedimmetadata!(x::TestMeta, dim::Int, key::AbstractString) | ||||||||||||||
delete!(x.dims[dim], key) | ||||||||||||||
return x | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
function DataAPI.emptydimmetadata!(x::TestMeta, dim::Int) | ||||||||||||||
Tokazama marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
empty!(x.dims[dim]) | ||||||||||||||
return x | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
|
||||||||||||||
# An example implementation of a table (without the Tables.jl interface) | ||||||||||||||
# for testing DataAPI.rownumber | ||||||||||||||
struct TestTable{T} | ||||||||||||||
|
@@ -301,12 +345,23 @@ end | |||||||||||||
@test_throws MethodError DataAPI.colmetadatakeys(1, 1) | ||||||||||||||
@test_throws MethodError DataAPI.colmetadatakeys(1) | ||||||||||||||
|
||||||||||||||
@test_throws MethodError DataAPI.dimmetadata!(1, 1, "a", 10, style=:default) | ||||||||||||||
@test_throws MethodError DataAPI.deletedimmetadata!(1, 1, "a") | ||||||||||||||
@test_throws MethodError DataAPI.emptydimmetadata!(1, 1) | ||||||||||||||
@test_throws MethodError DataAPI.dimmetadata(1, 1, "a") | ||||||||||||||
@test_throws ArgumentError DataAPI.dimmetadata(1, 1) | ||||||||||||||
@test_throws MethodError DataAPI.dimmetadata(1, 1, "a", style=true) | ||||||||||||||
@test_throws ArgumentError DataAPI.dimmetadata(1, 1, style=true) | ||||||||||||||
|
||||||||||||||
@test DataAPI.metadatasupport(Int) == (read=false, write=false) | ||||||||||||||
@test DataAPI.colmetadatasupport(Int) == (read=false, write=false) | ||||||||||||||
@test DataAPI.dimmetadatasupport(Int, 1) == (read=false, write=false) | ||||||||||||||
|
||||||||||||||
tm = TestMeta() | ||||||||||||||
@test DataAPI.metadatasupport(TestMeta) == (read=true, write=true) | ||||||||||||||
@test DataAPI.colmetadatasupport(TestMeta) == (read=true, write=true) | ||||||||||||||
@test DataAPI.dimmetadatasupport(TestMeta, 1) == (read=true, write=true) | ||||||||||||||
@test DataAPI.dimmetadatasupport(TestMeta, 0) == (read=false, write=false) | ||||||||||||||
|
||||||||||||||
@test isempty(DataAPI.metadatakeys(tm)) | ||||||||||||||
@test DataAPI.metadata(tm) == Dict() | ||||||||||||||
|
@@ -353,6 +408,28 @@ end | |||||||||||||
@test DataAPI.colmetadata!(tm, :col, "a", "100", style=:note) == tm | ||||||||||||||
DataAPI.emptycolmetadata!(tm) | ||||||||||||||
@test isempty(DataAPI.colmetadatakeys(tm)) | ||||||||||||||
|
||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm, 1)) | ||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm)[1]) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1) == Dict() | ||||||||||||||
@test DataAPI.dimmetadata(tm) == (Dict(), Dict()) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, style=true) == Dict() | ||||||||||||||
@test DataAPI.dimmetadata!(tm, 1, "a", "100", style=:note) == tm | ||||||||||||||
@test collect(DataAPI.dimmetadatakeys(tm, 1)) == ["a"] | ||||||||||||||
@test_throws KeyError DataAPI.dimmetadata(tm, 1, "b") | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "b", 123) == 123 | ||||||||||||||
@test_throws KeyError DataAPI.dimmetadata(tm, 1, "b", style=true) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "b", 123, style=true) == (123, :default) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "a") == "100" | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1) == Dict("a" => "100") | ||||||||||||||
@test DataAPI.dimmetadata(tm)[1] == Dict("a" => "100") | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, "a", style=true) == ("100", :note) | ||||||||||||||
@test DataAPI.dimmetadata(tm, 1, style=true) == Dict("a" => ("100", :note)) | ||||||||||||||
DataAPI.deletedimmetadata!(tm, 1, "a") | ||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm, 1)) | ||||||||||||||
@test DataAPI.dimmetadata!(tm, 1, "a", "100", style=:note) == tm | ||||||||||||||
DataAPI.emptydimmetadata!(tm, 1) | ||||||||||||||
@test isempty(DataAPI.dimmetadatakeys(tm, 1)) | ||||||||||||||
end | ||||||||||||||
|
||||||||||||||
@testset "rownumber" begin | ||||||||||||||
|
Uh oh!
There was an error while loading. Please reload this page.