-
Notifications
You must be signed in to change notification settings - Fork 7
Add 2.11 release post #6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
1c7d663
Add 2.11 release post
joshmoore f0d4072
Update Release 2.11.0 Blog
sanketverma1704 b09dc36
Update-II Release 2.11.0 Blog
sanketverma1704 4d7e309
Add chunk benchmark
joshmoore 1e4e277
Minor adjustments
joshmoore 62f18ed
Add vindex snippet
joshmoore 50ad47d
Use header for 'Appreciation' as well
joshmoore 63026a2
Minor fix
sanketverma1704 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,100 @@ | ||
--- | ||
layout: post | ||
title: "Zarr Python 2.11 Release" | ||
date: 2022-02-07 | ||
categories: blog | ||
permalink: /release-2-11/ | ||
--- | ||
|
||
Version 2.11 of the [Python Zarr package](https://zarr.readthedocs.io/en/stable/) has just been released. 🎉 | ||
|
||
This blog post aims to provide an overview of new features in this release, especially a new parameter that may impact the performance of writing arrays. | ||
|
||
|
||
## Empty chunks will no longer be written by default | ||
|
||
One of the advantages of the Zarr format is that it is sparse, which means that | ||
chunks with no data (more precisely, with data equal to the fill value, which | ||
is usually 0) don't need to be written to disk at all. They will simply be | ||
assumed to be empty at read time. However, until this release, the Zarr library | ||
would write these empty chunks to disk anyway. This changes in this version: a | ||
small performance penalty at write time leads to significant speedups at read | ||
time and in filesystem operations in the case of sparse arrays. To revert to | ||
the old behaviour, pass the argument ``write_empty_chunks=True`` to the array | ||
creation function. | ||
|
||
This feature was added by [Davis Bennett](https://github.com/jni) and [Juan Nunez-Iglesias](https://github.com/jni) | ||
with PR [#738](https://github.com/zarr-developers/zarr-python/issues/738) and [#853](https://github.com/zarr-developers/zarr-python/issues/853) respectively. | ||
|
||
Some preliminary benchmark results are shown in PR [#853](https://github.com/zarr-developers/zarr-python/pull/853#issuecomment-988245713). For example: | ||
|
||
 | ||
|
||
shows how with the previous setting of `write_empty_chunks=True` the speed of | ||
writing a fill-valued chunk is the same as writing a randomly-filled chunk. | ||
With the new default of `write_empty_chunks_False`, writing the fill-valued | ||
chunk is *much* faster while writing a randomly-filled chunk is slightly | ||
slower. | ||
|
||
## Fancy numpy-style indexing | ||
|
||
Zarr arrays now support NumPy-style fancy indexing with arrays of integer | ||
coordinates. This is equivalent to using ``zarr.Array.vindex``. Mixing slices and | ||
integer arrays are not supported. | ||
|
||
``` | ||
>>> z.vindex[[0, 2], [1, 3]] | ||
array([-1, -2]) | ||
>>> z.vindex[[0, 2], [1, 3]] = [-3, -4] | ||
>>> z[:] | ||
array([[ 0, -3, 2, 3, 4], | ||
[ 5, 6, 7, 8, 9], | ||
[10, 11, 12, -4, 14]]) | ||
>>> z[[0, 2], [1, 3]] | ||
array([-3, -4]) | ||
``` | ||
|
||
See [Advanced indexing](https://zarr.readthedocs.io/en/stable/tutorial.html#advanced-indexing) in the tutorial for more information. | ||
|
||
This feature was added by [Juan Nunez-Iglesias](https://github.com/jni) with PR [#725](https://github.com/zarr-developers/zarr-python/issues/725). | ||
|
||
## New base class | ||
|
||
This release of Zarr Python introduces a new ``BaseStore`` class that all | ||
provided store classes implemented in Zarr Python now inherit from. This is | ||
done as part of refactoring to enable future support of the Zarr version 3 | ||
spec. Existing third-party stores that are a MutableMapping (e.g. dict) can be | ||
converted to a new-style key/value store inheriting from ``BaseStore`` by | ||
passing them as the argument to the new ``zarr.storage.KVStore`` class. For | ||
backwards compatibility, various higher-level array creation and convenience | ||
functions still accept plain Python dicts or other mutable mappings for the | ||
``store`` argument, but will internally convert these to a ``KVStore``. | ||
|
||
This feature was added by [Greggory Lee](https://github.com/grlee77) with PR | ||
[#839](https://github.com/zarr-developers/zarr-python/issues/839), | ||
[#789](https://github.com/zarr-developers/zarr-python/issues/789) and | ||
[#950](https://github.com/zarr-developers/zarr-python/issues/950). | ||
|
||
## More information | ||
|
||
Details on these features as well as the full list of all changes in 2.11.0 are available on the release notes. Check [here](https://zarr.readthedocs.io/en/stable/release.html#release-2-11-0). | ||
|
||
## Appreciation 🙌🏻 | ||
|
||
Shout-out to all the contributors who made release 2.11.0 possible: | ||
- [Juan Nunez-Iglesias](https://github.com/jni) | ||
- [Davis Bennett](https://github.com/d-v-b) | ||
- [Gregory Lee](https://github.com/grlee77) | ||
- [Ryan Abernathy](https://github.com/rabernat) | ||
- [Matthias Bussonnier](https://github.com/Carreau) | ||
- [Oren Watson](https://github.com/orenwatson) | ||
- [Joe Hamman](https://github.com/jhamman) | ||
- [Dimitri Papadopoulos Orfanos](https://github.com/DimitriPapadopoulos) | ||
- [Ray Bell](https://github.com/raybellwaves) | ||
- [John Kirkham](https://github.com/jakirkham) | ||
- [Mads R. B. Kristensen](https://github.com/madsbk) | ||
- [Josh Moore](https://github.com/joshmoore). | ||
|
||
If you find the above features useful and end up using them, please mention [@zarr_dev](https://twitter.com/zarr_dev) on Twitter and tweet using #ZarrData and we'll make sure to get it featured! ✌🏻 | ||
|
||
~Sanket Verma |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.