From 60a86b0a755a3a536158974c893b15277865aba5 Mon Sep 17 00:00:00 2001 From: jmoore Date: Mon, 7 Feb 2022 12:37:27 +0100 Subject: [PATCH 1/4] Release notes 2.11.0 --- .github/workflows/minimal.yml | 2 +- docs/release.rst | 119 +++++++++++++++++++++++++++++----- 2 files changed, 105 insertions(+), 16 deletions(-) diff --git a/.github/workflows/minimal.yml b/.github/workflows/minimal.yml index ff68783d33..eb6ebd5d25 100644 --- a/.github/workflows/minimal.yml +++ b/.github/workflows/minimal.yml @@ -9,7 +9,7 @@ on: branches: [ master ] jobs: - build: + minimum_build: runs-on: ubuntu-latest steps: diff --git a/docs/release.rst b/docs/release.rst index 698261fabe..3e06038f62 100644 --- a/docs/release.rst +++ b/docs/release.rst @@ -6,28 +6,117 @@ Release notes Unreleased ---------- -This release of Zarr Python introduces a new ``BaseStore`` class that all provided store classes implemented in Zarr Python now inherit from. This is done as part of refactoring to enable future support of the Zarr version 3 spec. Existing third-party stores that are a MutableMapping (e.g. dict) can be converted to a new-style key/value store inheriting from ``BaseStore`` by passing them as the argument to the new ``zarr.storage.KVStore`` class. For backwards compatibility, various higher-level array creation and convenience functions still accept plain Python dicts or other mutable mappings for the ``store`` argument, but will internally convert these to a ``KVStore``. +.. _release_2.11.0: +2.11.0 +------ Enhancements ~~~~~~~~~~~~ -* write_empty_chunks defaults to False. - By :user:`Juan Nunez-Iglesias `; :issue:`853`. +* **Fancy indexing**. Zarr arrays now support NumPy-style fancy indexing with + arrays of integer coordinates. This is equivalent to using zarr.Array.vindex. + Mixing slices and integer arrays is not supported. + By :user:`Juan Nunez-Iglesias `; :issue:`725`. -* Allow to assign array ``fill_values`` and update metadata accordingly. :issue:`662` -* array indexing with [] (getitem and setitem) now supports fancy indexing. - By :user:`Juan Nunez-Iglesias `; :issue:`725`. +* **Sparse changes!**. One of the advantages of the Zarr format is that it is + sparse, which means that chunks with no data (more precisely, with data equal + to the fill value, which is usually 0) don't need to be written to disk at + all. They will simply be assumed to be empty at read time. However, until + this release, the Zarr library would write these empty chunks to disk anyway. + This changes in this version: a small performance penalty at write time leads + to significant speedups at read time and in filesystem operations in the case + of sparse arrays. To revert to the old behavior, pass the argument + write_empty_chunks=True to the array creation function. + By :user:`Juan Nunez-Iglesias `; :issue:`853` and + :user:`Davis Bennett `; :issue:`738`. + +* **New base class**. This release of Zarr Python introduces a new + ``BaseStore`` class that all provided store classes implemented in Zarr + Python now inherit from. This is done as part of refactoring to enable future + support of the Zarr version 3 spec. Existing third-party stores that are a + MutableMapping (e.g. dict) can be converted to a new-style key/value store + inheriting from ``BaseStore`` by passing them as the argument to the new + ``zarr.storage.KVStore`` class. For backwards compatibility, various + higher-level array creation and convenience functions still accept plain + Python dicts or other mutable mappings for the ``store`` argument, but will + internally convert these to a ``KVStore``. + By :user:`Greggory Lee `; :issue:`839`, :issue:`789`, and :issue:`950`. + +* Allow to assign array ``fill_values`` and update metadata accordingly. + By :user:`Ryan Abernathey `, :issue:`662`. + +* Allow to update array fill_values + By :user:`Matthias Bussonnier ` :issue:`665`. + +Bug fixes +~~~~~~~~~ + +* Fix bug where the checksum of zipfiles is wrong + By :user:`Oren Watson ` :issue:`930`. + +* Fix consolidate_metadata with FSStore. + By :user:`Joe Hamman ` :issue:`916`. + +* Unguarded next inside generator. + By :user:`Dimitri Papadopoulos Orfanos ` :issue:`889`. + +* Fix: N5 keywords now emit UserWarning instead of raising a ValueError + By :user:`Boaz Mohar ` :issue:`860`. + +Documentation +~~~~~~~~~~~~~ + +* Update docs creation of dev env. + By :user:`Ray Bell ` :issue:`921`. -* write_empty_chunks=False deletes chunks consisting of only fill_value. - By :user:`Davis Bennett `; :issue:`738`. +* Update docs to use `python -m pytest`. + By :user:`Ray Bell ` :issue:`923`. -* Move metadata handling to a class. - By :user:`Greggory Lee `; :issue:`839`. +* Fix versionadded tag in zarr.core.Array docstring. + By :user:`Juan Nunez-Iglesias ` :issue:`852`. -* Create a Base store class for Zarr Store. - By :user:`Greggory Lee `; :issue:`789`. +* Doctest seem to be stricter now, updating tostring() to tobytes(). + By :user:`John Kirkham ` :issue:`907`. + +* Minor doc fix. + By :user:`Mads R. B. Kristensen ` :issue:`937`. + +Maintenance +~~~~~~~~~~~ + +* Upgrade MongoDB in test env. + By :user:`Joe Hamman ` :issue:`939`. + +* Define blocksize for array, updates hexdigest values. + By :user:`Andrew Fulton ` :issue:`867`. + +* blocks_to_decompress not used in read_part function. + By :user:`Boaz Mohar ` :issue:`861`. + +* Pass dimension_separator on fixture generation. + By :user:`Josh Moore ` :issue:`858`. + +* Activate Python 3.9 in GitHub Actions. + By :user:`Josh Moore ` :issue:`859`. + +* Drop shortcut `fsspec[s3]` for dependency. + By :user:`Josh Moore ` :issue:`920`. + +* and a swath of code-linting improvements by :user:`Dimitri Papadopoulos Orfanos `: + - Unnecessary comprehension (:issue:`899`) + - Unnecessary `None` provided as default (:issue:`900`) + - use an if `expression` instead of `and`/`or` (:issue:`888`) + - iRemove unnecessary literal (:issue:`891`) + - Decorate a few method with `@staticmethod` (:issue:`885`) + - Useless `return` (:issue:`884`) + - In Python 3, all classes implicitly inherit from object (:issue:`886`) + - Unnecessary comprehension (:issue:`883`) + - Codespell configuration (:issue:`882`) + - Fix typos found by codespell (:issue:`880`) + - Proper C-style formatting for integer (:issue:`913`) + - Add LGTM.com / DeepSource.io configuration files (:issue:`909`) .. _release_2.10.3: @@ -144,7 +233,7 @@ Maintenance ~~~~~~~~~~~ * Correct conda-forge deployment of Zarr. - By :user:`Josh Moore `; :issue:`XXX`. + By :user:`Josh Moore `; :issue:`819`. .. _release_2.9.0: @@ -346,8 +435,8 @@ This release of Zarr Python is the first release of Zarr to not support Python 3 `_ for some performance analysis showing order of magnitude faster response in some benchmark. -See `this link ` for the full list of closed and -merged PR tagged with the 2.6 milestone. +See `this link `_ +for the full list of closed and merged PR tagged with the 2.6 milestone. * Add ability to partially read and decompress arrays, see :issue:`667`. It is only available to chunks stored using fsspec and using Blosc as a compressor. From df9a34bf7ace28f2f37449cf8bf5cb5027211952 Mon Sep 17 00:00:00 2001 From: jmoore Date: Mon, 7 Feb 2022 12:43:08 +0100 Subject: [PATCH 2/4] Rendering improvements for changelog --- docs/release.rst | 48 ++++++++++++++++++++++++++++++------------------ 1 file changed, 30 insertions(+), 18 deletions(-) diff --git a/docs/release.rst b/docs/release.rst index 3e06038f62..0619a62faa 100644 --- a/docs/release.rst +++ b/docs/release.rst @@ -14,24 +14,24 @@ Unreleased Enhancements ~~~~~~~~~~~~ +* **Sparse changes with performance impact!** One of the advantages of the Zarr + format is that it is sparse, which means that chunks with no data (more + precisely, with data equal to the fill value, which is usually 0) don't need + to be written to disk at all. They will simply be assumed to be empty at read + time. However, until this release, the Zarr library would write these empty + chunks to disk anyway. This changes in this version: a small performance + penalty at write time leads to significant speedups at read time and in + filesystem operations in the case of sparse arrays. To revert to the old + behavior, pass the argument ``write_empty_chunks=True`` to the array creation + function. By :user:`Juan Nunez-Iglesias `; :issue:`853` and + :user:`Davis Bennett `; :issue:`738`. + + * **Fancy indexing**. Zarr arrays now support NumPy-style fancy indexing with arrays of integer coordinates. This is equivalent to using zarr.Array.vindex. Mixing slices and integer arrays is not supported. By :user:`Juan Nunez-Iglesias `; :issue:`725`. - -* **Sparse changes!**. One of the advantages of the Zarr format is that it is - sparse, which means that chunks with no data (more precisely, with data equal - to the fill value, which is usually 0) don't need to be written to disk at - all. They will simply be assumed to be empty at read time. However, until - this release, the Zarr library would write these empty chunks to disk anyway. - This changes in this version: a small performance penalty at write time leads - to significant speedups at read time and in filesystem operations in the case - of sparse arrays. To revert to the old behavior, pass the argument - write_empty_chunks=True to the array creation function. - By :user:`Juan Nunez-Iglesias `; :issue:`853` and - :user:`Davis Bennett `; :issue:`738`. - * **New base class**. This release of Zarr Python introduces a new ``BaseStore`` class that all provided store classes implemented in Zarr Python now inherit from. This is done as part of refactoring to enable future @@ -71,7 +71,7 @@ Documentation * Update docs creation of dev env. By :user:`Ray Bell ` :issue:`921`. -* Update docs to use `python -m pytest`. +* Update docs to use ``python -m pytest``. By :user:`Ray Bell ` :issue:`923`. * Fix versionadded tag in zarr.core.Array docstring. @@ -101,21 +101,33 @@ Maintenance * Activate Python 3.9 in GitHub Actions. By :user:`Josh Moore ` :issue:`859`. -* Drop shortcut `fsspec[s3]` for dependency. +* Drop shortcut ``fsspec[s3]`` for dependency. By :user:`Josh Moore ` :issue:`920`. * and a swath of code-linting improvements by :user:`Dimitri Papadopoulos Orfanos `: + - Unnecessary comprehension (:issue:`899`) - - Unnecessary `None` provided as default (:issue:`900`) - - use an if `expression` instead of `and`/`or` (:issue:`888`) + + - Unnecessary ``NoneP` provided as default (:issue:`900`) + + - use an if ``expression`` instead of `and`/`or` (:issue:`888`) + - iRemove unnecessary literal (:issue:`891`) + - Decorate a few method with `@staticmethod` (:issue:`885`) - - Useless `return` (:issue:`884`) + + - Useless ``return`` (:issue:`884`) + - In Python 3, all classes implicitly inherit from object (:issue:`886`) + - Unnecessary comprehension (:issue:`883`) + - Codespell configuration (:issue:`882`) + - Fix typos found by codespell (:issue:`880`) + - Proper C-style formatting for integer (:issue:`913`) + - Add LGTM.com / DeepSource.io configuration files (:issue:`909`) .. _release_2.10.3: From e6a77cb0959d4c7fc2eecb836a16876dd3435d50 Mon Sep 17 00:00:00 2001 From: jmoore Date: Mon, 7 Feb 2022 12:44:50 +0100 Subject: [PATCH 3/4] Remove duplicated entries from 2.10.3 --- docs/release.rst | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/docs/release.rst b/docs/release.rst index 0619a62faa..1902c94ff0 100644 --- a/docs/release.rst +++ b/docs/release.rst @@ -26,7 +26,6 @@ Enhancements function. By :user:`Juan Nunez-Iglesias `; :issue:`853` and :user:`Davis Bennett `; :issue:`738`. - * **Fancy indexing**. Zarr arrays now support NumPy-style fancy indexing with arrays of integer coordinates. This is equivalent to using zarr.Array.vindex. Mixing slices and integer arrays is not supported. @@ -62,9 +61,6 @@ Bug fixes * Unguarded next inside generator. By :user:`Dimitri Papadopoulos Orfanos ` :issue:`889`. -* Fix: N5 keywords now emit UserWarning instead of raising a ValueError - By :user:`Boaz Mohar ` :issue:`860`. - Documentation ~~~~~~~~~~~~~ @@ -89,12 +85,6 @@ Maintenance * Upgrade MongoDB in test env. By :user:`Joe Hamman ` :issue:`939`. -* Define blocksize for array, updates hexdigest values. - By :user:`Andrew Fulton ` :issue:`867`. - -* blocks_to_decompress not used in read_part function. - By :user:`Boaz Mohar ` :issue:`861`. - * Pass dimension_separator on fixture generation. By :user:`Josh Moore ` :issue:`858`. From 9fd78b545dc4f7b1128b0dcb00a2729b695ec862 Mon Sep 17 00:00:00 2001 From: Josh Moore Date: Mon, 7 Feb 2022 13:08:46 +0100 Subject: [PATCH 4/4] Apply suggestions from code review Co-authored-by: jakirkham --- docs/release.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/release.rst b/docs/release.rst index 1902c94ff0..16c884efc7 100644 --- a/docs/release.rst +++ b/docs/release.rst @@ -98,17 +98,17 @@ Maintenance - Unnecessary comprehension (:issue:`899`) - - Unnecessary ``NoneP` provided as default (:issue:`900`) + - Unnecessary ``None`` provided as default (:issue:`900`) - use an if ``expression`` instead of `and`/`or` (:issue:`888`) - - iRemove unnecessary literal (:issue:`891`) + - Remove unnecessary literal (:issue:`891`) - Decorate a few method with `@staticmethod` (:issue:`885`) - - Useless ``return`` (:issue:`884`) + - Drop unneeded ``return`` (:issue:`884`) - - In Python 3, all classes implicitly inherit from object (:issue:`886`) + - Drop explicit ``object`` inheritance from ``class``es (:issue:`886`) - Unnecessary comprehension (:issue:`883`)