`exp run`: `--set-params` forces YAML 1.2 (breaks code that uses 1.1)

[flippedcoder]

Bug Report

dvc exp run --set-param lr=0.000001

Here's the error:

Traceback (most recent call last):
  File "/Users/milecia/Repos/checkpoints-tutorial/train.py", line 123, in <module>
    main()
  File "/Users/milecia/Repos/checkpoints-tutorial/train.py", line 110, in main
    train(model, x_batch, y_batch, params["lr"], params["weight_decay"])
  File "/Users/milecia/Repos/checkpoints-tutorial/train.py", line 45, in train
    optimizer = torch.optim.Adam(model.parameters(), lr=lr,
  File "/Users/milecia/Repos/checkpoints-tutorial/.venv/lib/python3.9/site-packages/torch/optim/adam.py", line 36, in __init__
    if not 0.0 <= lr:
TypeError: '<=' not supported between instances of 'float' and 'str'

Description

The problem is that the yaml library only supports yaml 1.1 when DVC uses yaml1.2. This issue seems to happen when exponents are generated.

The data science community and the Python ecosystem in general is stuck in YAML1.1. We could try to be compatible by generating exponents with the dot at least, that way we could be compatible on both sides. I am not sure how straightforward the fix this will be though. (and this should only affect --set-param, not the dvc.lock and yaml files).

Reproduce

1. Go to the checkpoints tutorial repo
2. Replace from ruamel.yaml import YAML with import yaml
3. Replace these 2 lines yaml=YAML(typ='safe') params = yaml.load(f) with params = yaml.safe_load(f)
4. dvc exp run --set-param lr=0.000001
5. The lr in params.yaml should be updated to an exponent and you get the error above

Expected

Running dvc exp run --set-param lr=0.000001 should start running the experiment with the new lr

Environment information

Output of dvc doctor:

DVC version: 2.1.0 (brew)
---------------------------------
Platform: Python 3.9.4 on macOS-11.3-x86_64-i386-64bit
Supports: azure, gdrive, gs, http, https, s3, ssh, oss, webdav, webdavs

[pmrowla]

This is by design and is documented: https://dvc.org/doc/command-reference/metrics#supported-file-formats (yaml 1.2 is also specified for plots and params)

See also #4281 and the associated PR

---

Regarding the checkpoints tutorial, we should really be using ruamel and not PyYAML in any of the iterative example repos for this specific reason.

[dberenbaum]

@skshetry I know you suggested this issue. Was there something you had in mind?

Edit: I missed your response in Slack, which I'm pasting below.

@dave Berenbaum, it's more of a compatibility issue. The data science community and the python ecosystem in general is stuck in YAML1.1. We could try to be compatible by generating exponents with the dot at least, that way we could be compatible on both sides. I am not sure how straightforward the fix this will be though. (and this should only affect --set-param, not the dvc.lock and yaml files).

[dberenbaum]

Also related to #5777. Both issues seem to stem from dvc parsing the provided values instead of replacing them as is.

[dberenbaum]

The data science community and the Python ecosystem in general is stuck in YAML1.1.

Agreed, and it's not obvious when you get the error that it's a YAML 1.1/1.2 issue. We can change the tutorials, but it doesn't help users who run into the same error.

[dmpetrov]

We could try to be compatible by generating exponents with the dot at least, that way we could be compatible on both sides.

That seems like a viable solution.

We can change the tutorials, but it doesn't help users who run into the same error.

Yes. The tutorials can help mitigate the issue but it isa not a solution. A good portion of users will get the error even the ones who do not use Python at all.

[jorgeorpinel]

DVC is not compatible with yaml 1.1

This is by design and is documented

Agree. Maybe the problem (or part of) begins with the crash and tracelog, which seems to indicate that the error happened in the user code. I also wonder what it would look like if the user code wasn't Python...

Could we start by catching it somehow and presenting a meaningful error message like ERROR: Please update your 'param.yam' file to YAML 1.2 ? To my mind that would remove the "bug" part from this issue.

[pmrowla]

The error here is happening in the (example) user code - the params.yaml is already YAML 1.2, the problem is that the user tries to open it with PyYAML (which only supports 1.1).

[jorgeorpinel]

I see. I thought this only happened when the user code is run via exp run (not manually, for example).

Yeah nvmd I got it now: exp run -P makes the params file YAML 1.2, which causes the cmd to fail.