Closed
Description
According to our current implementation of B-trees:
rust/src/liballoc/btree/node.rs
Lines 53 to 55 in 5965b79
it would appear that up to
11
key-value pairs can be stored in each node.
For u128
values representing 128
set elements each, 1408
set elements can be stored in a single allocation, with an overhead of around 50% compared to Vec<u128>
in the dense case.
Such sparse bitsets would be really useful for (e.g. dataflow) analysis algorithms, in situations where the bitset elements tend to be localized, with multi-"word" gaps in between local groups.
Metadata
Metadata
Assignees
Labels
Area: `std::collections`Category: An issue proposing an enhancement or a PR with one.Issue: Problems and improvements with respect to memory usage during compilation.Issue: Problems and improvements with respect to compile times.Relevant to the compiler team, which will review and decide on the PR/issue.
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
nikomatsakis commentedon Jan 19, 2018
Is the idea that the key would be
index >> 7
and the value would be a bitset of size 128? i.e., so to tell if something is there, we compute the key and then mask the resulting value?eddyb commentedon Jan 19, 2018
@nikomatsakis That's pretty much it, yes. Here's what I've been playing with so far:
nikomatsakis commentedon Jan 19, 2018
Makes sense. I wonder if it would be useful in NLL too.
eternaleye commentedon Jan 19, 2018
One thing that might be worth considering is using binmaps.
eddyb commentedon Jan 22, 2018
We should also benchmark "sparse bit matrices" with
BTreeMap
(orFxHashMap
) keys that also contain the "row" number, not just the upper bits of the "column" number.They might prove more efficient than
Vec<BTreeMap<u32, u128>>
.EDIT: This makes iteration within a "row" harder, especially with the
HashMap
variant.eddyb commentedon Feb 21, 2018
I've added a "chunked" API to my
SparseBitSet
, in the hope of 128x+ perf wins:cc @spastorino
spastorino commentedon Feb 23, 2018
Done here #48245
tamird commentedon Mar 1, 2018
Should this be closed, now?
9 remaining items