Wait for dvc lock using sleep cycle

[prihoda]

Currently, we are waiting for the dvc lock file for 5 seconds and then fail if it is still not available. Would there be an issue with waiting for the lock using a sleep cycle with a defined refresh delay (~2s) and timeout (~2min or configurable)? Or are there some other long-term plans that would solve this problem?

This would be especially useful for the IntelliJ plugin or any other tool that calls DVC programmatically.

[efiop]

Might be missing something, but how is it different from just adjusting timeout from 5 seconds to 2 minutes?

[prihoda]

Well if I understand it correctly, the _do_lock method will throw the LockError exception immediately if the lock is not available. Now we catch the LockError for the first time it happens, sleep for the provided timeout and then try again, this time failing for good if the lock is not available:

try:
    self._do_lock()
    return
except LockError:
    time.sleep(self.TIMEOUT)

self._do_lock()

[efiop]

Ah, you are right. For some reason, I've thought that we had a "true" timeout there. Thanks!

[prihoda]

Anyway, from the IntelliJ plugin's side, I realized that at this moment it might actually be better to keep a queue of DVC commands and make sure that we are processing it sequentially. The only problem is that if we implement some smarter lock method that allows some commands to be run in parallel (e.g. using per-stage locking), the plugin would have to be aware of which commands can run in parallel and which can not.

[efiop]

@prihoda that would be a great way to go about it, especially considering that some commands like repro might run for longer than a pre-set timeout, so you would not be able to choose that timeout correctly beforehand.