[WIP] cache/remote: drop dos2unix MD5 by default #5337
Conversation
- LocalCache tree root is repo root, but ODB root should be .dvc/cache
| return schema[version] | ||
|
|
||
|
|
||
| class BaseODB: |
There was a problem hiding this comment.
As it is now this is really more of an ODBConfig class, but given that nothing else will probably get refactored into ODB for 2.0 I did not spend too much time thinking about how everything should be organized in the long run
There was a problem hiding this comment.
@pmrowla, is this later going to evolve and be used in chunking?
There was a problem hiding this comment.
@skshetry yes, that's the plan for now at least
There was a problem hiding this comment.
Maybe then just ObjectDB?
| @@ -450,6 +452,7 @@ def create_taskset(amount): | |||
|
|
|||
| @index_locked | |||
| def push(self, cache, named_cache, jobs=None, show_checksums=False): | |||
| self.odb.migrate_config() | |||
There was a problem hiding this comment.
Updating/writing the config should really be done as needed after an initial odb.write/save/etc operation, but since we don't do anything else with ODB at the moment writing the config file to a remote on push should be sufficient for 2.0
| @@ -90,6 +90,10 @@ def hash_jobs(self): | |||
| or self.HASH_JOBS | |||
| ) | |||
|
|
|||
| @cached_property | |||
| def enable_dos2unix(self): | |||
| return self.repo and self.repo.config["core"].get("dos2unix") | |||
There was a problem hiding this comment.
I don't think this really belongs in BaseTree long term, but currently we compute MD5's from both RepoTree and LocalTree
| def test_caches(tmp_dir, dvc, caplog, mocker): | ||
| from dvc.tree.ssh import SSHTree | ||
|
|
||
| mocker.patch.object(SSHTree, "exists", return_value=False) |
There was a problem hiding this comment.
now that we will try to load a config file on remote connection, this has to be mocked out for these kinds of tests with a fake remote URL otherwise the test will hang until the network call times out
pmrowla
changed the title
Checkout is an operation that builds object from cache(aka object database) in a particular tree(aka filesystem). Thus it shouldn't be part of the object database itself nor of the filesystem. Pre-requisite for iterative#5337
Checkout is an operation that builds object from cache(aka object database) in a particular tree(aka filesystem). Thus it shouldn't be part of the object database itself nor of the filesystem. Pre-requisite for #5337
This makes all cache objects go through the same logic, so that all of the appropriate rules are applied to them (e.g. proper permissions). Pre-requisite for iterative#5337
This makes all cache objects go through the same logic, so that all of the appropriate rules are applied to them (e.g. proper permissions). Pre-requisite for #5337
|
Will refactor this into a new PR after @efiop's recent cache/object changes |
❗ I have followed the Contributing to DVC checklist.
📖 If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.
Thank you for the contribution - we'll try to review it as soon as possible. 🙏
core.dos2unixto be true2.0ODB schema version (the only migration step taken at this time is to write the config/version file)Will close #4658.
Related to #829.