Mutable mapping for Azure Blob #345

shikharsg · 2018-11-30T08:15:41Z

[Description of PR]

TODO:

Add unit tests and/or doctests in docstrings
Add docstrings and API docs for any new/modified user-facing classes and functions
New/modified features documented in docs/tutorial.rst
Changes documented in docs/release.rst
Docs build locally (e.g., run tox -e docs)
AppVeyor and Travis CI passes
Test coverage is 100% (Coveralls passes)

Was missing self.container_name as an argument

Join only accepts one argument, using os.path.join(x,y) formats the string as a valid file path for us.

Quick fix for small bugs in ABSStore class in storage.py

Add ABSStore

…h python 3.5

alimanfoo · 2019-02-12T17:29:26Z

The travis build is not being triggered on the latest commits that I am making. So if you see the requests page of the travis build here, for some of the commits in this pull request it says 'Branch "abs_store" not included per configuration.', so I thought if I could add the abs_store branch to the travis.yml file(temporarily of course) then the build will be triggered. Like this:
branches:
  only:
    - master
    - abs_store

That shouldn't be necessary. Perhaps travis is getting confused because you also have a PR open from your branch into zarr-developers:abs_store (#293). Can #293 be closed now in favour of this PR?

shikharsg · 2019-02-12T20:05:12Z

@alimanfoo thanks, that worked

shikharsg · 2019-02-13T11:45:34Z

I have added a small snippet to tutorial.rst, but that's still using the emulator. So we can choose whether or not to keep that.

Apart from that I think this is good to go.

jakirkham · 2019-02-18T00:10:16Z

.travis.yml

@@ -24,6 +25,10 @@ matrix:
      dist: xenial
      sudo: true

+before_install:
+  - docker pull arafato/azurite


In the future, this may move to an azure org on Docker Hub ( Azure/Azurite#94 ), but it hasn't happened yet. Happy with leaving this as is. Just raising awareness.

thanks for pointing out

zarr/storage.py

jakirkham

Couple of minor comments above. Otherwise LGTM.

shikharsg · 2019-02-21T15:56:05Z

Thanks @jakirkham, I have addressed your comments

dazzag24 · 2019-02-22T15:19:13Z

Hi,
Is there anything else required on this pull request? It'd be great to get this into master and into a release.
Thanks
Darren

jakirkham · 2019-02-22T23:02:48Z

I think some other people that have helped in the review (or have not) should give it a look to see if everything meets their expectations.

dazzag24 · 2019-02-28T15:13:57Z

Any further input from anyone? I'm keen to merge

alimanfoo

This all looks good to me. Spotted a couple of very minor documentation fixes, but otherwise more than happy to merge.

docs/release.rst

zarr/storage.py

Co-Authored-By: shikharsg <[email protected]>

jakirkham · 2019-03-01T03:42:37Z

Thanks all :)

alimanfoo · 2019-03-01T08:28:50Z

Thanks indeed, fantastic to see this go in 👏

I just realised that I missed @mzjp2 in the credits for this feature, I'll update the release notes directly.

dazzag24 · 2019-03-01T09:15:33Z

Thanks all!

shikharsg · 2019-03-01T09:31:31Z

Thanks all!

rabernat and others added 30 commits August 2, 2018 15:18

basic implementation working

fd5bfb5

docs and cleanup

a26752c

fixed client_kwargs bug

23dd8f6

Add ABSStore mutable mapping

dec75dd

Fix import syntax

13c2077

Get open_zarr() working

86603fa

Change account variable names

4b999ba

Fix client.exists() logging issue

677ec1c

Minor comment changes

d9be9ba

Get to_zarr() working

13b1ee8

Remove state['container'] delete

e5564c3

Implement rmdir

a85e559

Add docstring for ABSStore

272d234

Remove GCSStore from this branch

bb406a0

Fixed missing argument in getsize of ABStore

937d162

Was missing self.container_name as an argument

Specified prefix argument in rmdir for abstore

74920c4

Fixed join string error in dir_path in ABStore

bd1648b

Join only accepts one argument, using os.path.join(x,y) formats the string as a valid file path for us.

Merge pull request #2 from mzjp2/abs_store

e79cb4f

Quick fix for small bugs in ABSStore class in storage.py

Remove logging work-around as the issue was fixed in azure-storage 1.3.0

0e71f70

Clean up docstring

de5bb9c

Remove more GCSStore code

13a6d30

Move utility functions into ABSStore class

7b52e39

Merge pull request zarr-developers#288 from friedrichknuth/abs_store

29a8697

Add ABSStore

implemented the rest of the mutable mapping functions. tests pass wit…

36139cb

…h python 3.5

using local blob emulator for storage.ABSStore testing

bda0c3f

fixed PY2 array.array error in storage.ABSStore

447c473

create test container if not exists in ABSStore test

c6858ed

added more tests for ABSStore

8e51b3b

reverted blob client creation to inside of ABSStore

ec4e3f1

added group test for ABSStore

bde7b5e

shikharsg added 2 commits February 12, 2019 18:56

small change to docs

3a4f4d9

Merge branch 'abs_store' of github.com:shikharsg/zarr into abs_store

1da44b1

jakirkham reviewed Feb 18, 2019

View reviewed changes

zarr/storage.py Outdated Show resolved Hide resolved

jakirkham reviewed Feb 18, 2019

View reviewed changes

shikhar added 2 commits February 21, 2019 14:24

cleaned create blob code

b51fb78

flake8 fix

4af5ebe

dazzag24 mentioned this pull request Feb 22, 2019

2.3 release planning #339

Closed

jakirkham approved these changes Feb 22, 2019

View reviewed changes

alimanfoo approved these changes Feb 28, 2019

View reviewed changes

docs/release.rst Outdated Show resolved Hide resolved

zarr/storage.py Outdated Show resolved Hide resolved

zarr/storage.py Outdated Show resolved Hide resolved

alimanfoo self-assigned this Feb 28, 2019

alimanfoo and others added 2 commits March 1, 2019 00:44

Update docs/release.rst

13a7dc4

Co-Authored-By: shikharsg <[email protected]>

Apply suggestions from code review

a062e18

Co-Authored-By: shikharsg <[email protected]>

jakirkham merged commit 4c7b071 into zarr-developers:master Mar 1, 2019

This was referenced Mar 1, 2019

Pull of ABSStore #287

Closed

Add ABSStore #288

Merged

jakirkham added this to the v2.3 milestone Mar 1, 2019

shikharsg deleted the abs_store branch March 1, 2019 09:26

jakirkham mentioned this pull request Apr 16, 2019

Implementing MutableMapping for blob stores Azure/azure-storage-python#557

Open

Uh oh!

Mutable mapping for Azure Blob #345

Mutable mapping for Azure Blob #345

Uh oh!

Conversation

shikharsg commented Nov 30, 2018

Uh oh!

alimanfoo commented Feb 12, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shikharsg commented Feb 12, 2019

Uh oh!

shikharsg commented Feb 13, 2019

Uh oh!

jakirkham Feb 18, 2019

Choose a reason for hiding this comment

Uh oh!

shikharsg Feb 21, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jakirkham left a comment

Choose a reason for hiding this comment

Uh oh!

shikharsg commented Feb 21, 2019

Uh oh!

dazzag24 commented Feb 22, 2019

Uh oh!

jakirkham commented Feb 22, 2019

Uh oh!

dazzag24 commented Feb 28, 2019

Uh oh!

alimanfoo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jakirkham commented Mar 1, 2019

Uh oh!

alimanfoo commented Mar 1, 2019

Uh oh!

dazzag24 commented Mar 1, 2019

Uh oh!

shikharsg commented Mar 1, 2019

Uh oh!

Uh oh!

alimanfoo commented Feb 12, 2019 •

edited

Loading