cache: checksum collision if there is dos2unix in the pipeline

[efiop]

As @shcheklein noticed, if user will have dvc add data && dvc run -d data -o data.unix dos2unix data data.unix in his pipeline, we will get a cache collision since we use dos2unix to compute checksum in the first place.

@efiop , @shcheklein , should we this to our documentation? or is there any other way to address this issue?

[shcheklein]

@MrOutis I think the idea was that we'll migrate to a regular md5 calculation as part of the 1.0 release since it requires breaking the compatibility. May be we can migrate to sha1 as was discussed in #1568 along the way. What do think @efiop ?

[shcheklein]

It is also related to this one iterative/dvc.org#68

[efiop]

@shcheklein well, the problem with crlf would arise in any checksum, so it won't be solved by migrating to sha1 or any other algorithm. Most likely, we'll simply need to introduce an additional suffix that would mark that that file is dos2unix one. Need to take a closer look at it sometime.

@MrOutis As @shcheklein noticed, we already have an issue for it at iterative/dvc.org#68 , so we might eventually get to writing a doc for it in the future.

[shcheklein]

@efiop yep, I understand the crlf issue. My point was if we go with this change (switch to a regular md5) and decide to break compatibility, maybe it makes sense to migrate to sha1 or whatnot along the way.