os: make Rename atomic on Windows

[gopherbot]

by MartyMacGyver:

As described in bug 3366 (https://golang.org/issue/3366) the
os.Rename() function doesn't work in Windows in a way that is consistent with other
OSes. In Posix-compliant OSes, the os.Rename() function works properly (and atomically),
including when replacing an existing file. In Windows this does not work correctly as
designed and fails when replacing an existing file. Since it has been decided that this
functionality for Windows will not be implemented in the existing os.Rename(), it is
desirable to have an equivalent os.Replace() function that works across platforms. In
the case of non-Windows platforms it would be a simple wrapper for os.Rename(), but in
the case of Windows it would call the proper OS-level APIs to perform the operation in
an equivalent fashion.

Other languages have encountered and successfully tackled this same problem in Windows.
As one example, the report of the same issue in Python led to a detailed discussion and
ultimately had a successful resolution (http://bugs.python.org/issue8828). The
workaround used in other Go projects follows the same pattern - effectively creating an
os.Replace() function for this circumstance, using the Windows MoveFileEx API for
Windows platforms and falling through to os.Rename() for the others.

Since os.Rename() cannot be enhanced to handle Windows cases in an equivalent fashion,
an os.Replace() function that satisfies the same criteria would prevent developers from
re-inventing the same wheel in very similar ways way simply to avoid the disjoint
functionality in os.Rename() when it comes to this relatively common use case.

[ianlancetaylor]

Comment 1:

Should os.Replace handle the EXDEV case on Unix systems?  (On Unix os.Rename will fail
when trying to move a file to a different mounted file system).

Labels changed: added repo-main, release-go1.5.

Status changed to Accepted.

[gopherbot]

Comment 2 by MartyMacGyver:

I don't think that (at this point) os.Replace() should have extended functionality
beyond what os.Rename offers (e.g., os.Rename() isn't supposed to work across
filesystems). Atomicity with os.Replace() varies with filesystem - though it would be
implicit for Posix when using the fallthrough to os.Rename(), Windows requires more
finesse. The major functional change versus os.Rename() is handling Windows properly,
where one would use the MoveFileEx API *without* the MOVEFILE_COPY_ALLOWED flag to
maintain parity with Posix-style rename.
For the sake of atomicity I understand why this can't all be done in os.Rename(), but
os.Replace() is meant to loosen that guarantee for select OSes, not remove it. If kept
simple, (and documented as such), devs will know that on Posix hosts this will be
exactly the same as os.Rename, and on Windows hosts this will do its best but is
inherently non-Posix in nature (versus doing nothing at all in the case of such
non-Posix hosts).
** When one starts adding functionality such as the ability to move across filesystems
it quickly complicates things while breaking the guarantee of atomicity across the
board. If one wants cross-filesystem functionality (where atomic (or nearly-atomic)
behavior is not expected at all), one would use os.Copy() or os.Move() - which are
notable in their absence in Go. That's another wheel that people seem to reinvent a lot,
but it's outside of the scope of this issue except for the point that once you start
adding that level of functionality you are effectively creating those functions as well.
Better to keep that extended effort separate.

[anacrolix]

Comment 3:

What package and function name do you propose?

[gopherbot]

Comment 4 by MartyMacGyver:

Package os, function signature the same as for Rename(), e.g.,
    func Replace(oldpath, newpath string) error