POC: Sketch of localfilesystem and commiting

[martindurant]

@mariusvniekerk , suggesting here is where you might want to work something out.
This contains minimal stuff for method aliasing (simplest possible), an actual example file-system for local work, and a hacky "transaction" that writes to temporary files, moving to final destination when the transaction is over.
As usual, everything I happened to code here is up for altering or criticism.

In [7]: with fs.transaction():
   ...:     with fs.open('./temp', 'wb') as f:
   ...:         f.write(b'hi')
   ...:     print(fs.ls('.'))
   ...: print(fs.ls('.'))
   ...:
   ...:
[]
['/Users/mdurant/temp/temp']

[martindurant]

(and a prod to make the aliases work with docstrings, please :) )

[mariusvniekerk]

Yeah i think that would probably work as a minimal example.

I think the interesting thing for distributed filesystems would be to consider making the actual transaction class itself a thing that can be moved around.

This is easily achievable by implementers by making fs.transaction() return an object with a __enter__ and __exit__. Though i guess we can already serialize the AbstractFilesystem instead.

Not entirely sure if we want to be as strict by default?

[martindurant]

Should autocommit be necessarily True if _intrans isn't set, and should we avoid collecting up instances that might never be cleared?

[mariusvniekerk]

autocommit only applies to writing at __exit__ so i think its a sane default in terms of user expectations

[mariusvniekerk]

Thats more explicit. 👍

[martindurant]

OK, going to merge this now for further iteration. Of course, it does not yet explicitly deal with distributed transactions, since the file objects within a Transaction are not necessarily pickleable (but they should be, in this context) - however, you would be able to call FileSystem.current().transaction.complete(True|False) within any worker.