Skip to content

Use iterative algorithms for mkdir_recursive and rmdir_recursive #12573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 13, 2014
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 85 additions & 11 deletions src/libstd/io/fs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ fs::unlink(&path);

use c_str::ToCStr;
use clone::Clone;
use container::Container;
use iter::Iterator;
use super::{Reader, Writer, Seek};
use super::{SeekStyle, Read, Write, Open, IoError, Truncate,
Expand All @@ -62,6 +63,7 @@ use result::{Ok, Err};
use path;
use path::{Path, GenericPath};
use vec::{OwnedVector, ImmutableVector};
use vec_ng::Vec;

/// Unconstrained file access type that exposes read and write operations
///
Expand Down Expand Up @@ -528,10 +530,25 @@ pub fn mkdir_recursive(path: &Path, mode: FilePermission) -> IoResult<()> {
if path.is_dir() {
return Ok(())
}
if path.filename().is_some() {
try!(mkdir_recursive(&path.dir_path(), mode));

let mut comps = path.components();
let mut curpath = path.root_path().unwrap_or(Path::new("."));

for c in comps {
curpath.push(c);

match mkdir(&curpath, mode) {
Err(mkdir_err) => {
// already exists ?
if try!(stat(&curpath)).kind != io::TypeDirectory {
return Err(mkdir_err);
}
}
Ok(()) => ()
}
}
mkdir(path, mode)

Ok(())
}

/// Removes a directory at this path, after removing all its contents. Use
Expand All @@ -542,16 +559,47 @@ pub fn mkdir_recursive(path: &Path, mode: FilePermission) -> IoResult<()> {
/// This function will return an `Err` value if an error happens. See
/// `file::unlink` and `fs::readdir` for possible error conditions.
pub fn rmdir_recursive(path: &Path) -> IoResult<()> {
let children = try!(readdir(path));
for child in children.iter() {
if child.is_dir() {
try!(rmdir_recursive(child));
} else {
try!(unlink(child));
let mut rm_stack = Vec::new();
rm_stack.push(path.clone());

while !rm_stack.is_empty() {
let children = try!(readdir(rm_stack.last().unwrap()));
let mut has_child_dir = false;

// delete all regular files in the way and push subdirs
// on the stack
for child in children.move_iter() {
// FIXME(#12795) we should use lstat in all cases
let child_type = match cfg!(windows) {
true => try!(stat(&child)).kind,
false => try!(lstat(&child)).kind
};

if child_type == io::TypeDirectory {
rm_stack.push(child);
has_child_dir = true;
} else {
// we can carry on safely if the file is already gone
// (eg: deleted by someone else since readdir)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is the reason for including this, I'm not entirely convinced that it's necessary at all. I would hope that one of the removers would receive an error in this case.

Can you switch this back to using try!?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexcrichton Why? The only reasonable action to take if this fails for FileNotFound is to retry the deletion, which means there was no reason to bother returning the error in the first place.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have yet to see an explanation of why this should specifically check for only file not found errors. This makes no sense to me and I don't understand why it should keep going for this one kind of error, but bail out on all the others.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because removing the file is what we're trying to do. If the file was already removed, then someone did our work for us. But that's not a reason to return an error.

In a straight-up rm() method that removes only the listed path, returning FileNotFound makes sense. We wanted to perform a single action, and we couldn't perform that action.

But in rmdir_recursive(), we're trying to perform all actions necessary to delete a given directory. If we determine that we need to delete a file, and then discover the file was already deleted, returning an error doesn't make sense because we were never told to explicitly delete that file. That was something we told ourselves to do, in order to fulfill the desired action. Since we told ourselves to do it, we also handle the error ourselves.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a normal occurrence, this is not something that should be swept under the rug. Just a few milliseconds earlier I listed the directory and found files A, B, and C. I successfully deleted A, and then B. Turns one someone else deleted C in the meantime.

That is not normal. That should be delivered upstairs to let them deal with it. Just because our job happens to be done for us doesn't mean that we should continue along our merry way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upstairs didn't even know that C existed. Telling them that we didn't find C isn't something they can reasonably deal with. As I said before, the only reasonable action they have to take is to request the deletion of the directory a second time (although more likely they're just going to wrap this in a try!() block and end up passing this error even higher up).

If the path that was passed to rmdir_recursive() ends up returning FileNotFound when that is ultimately deleted, I can see returning that error (although personally I have no problem treating that as success as well; the path existed when we first called readdir() so this wasn't a case of the caller giving us bad input). But for nested files that the caller did not explicitly mention, I do not see any value in treating that as an error.

It's also worth noting that FileNotFound doesn't even provide a way to indicate that the file that was missing was not the one that was passed to the function. The only available location to record that info is in the IoError.detail, but that's an unstructured string, presumably intended to display to the user, rather than something we can use to record this information. If I call rmdir_recursive() and get FileNotFound back, I'm certainly going to expect that means the directory didn't exist, and be quite surprised when it turns out to still exist.

match unlink(&child) {
Ok(()) => (),
Err(ref e) if e.kind == io::FileNotFound => (),
Err(e) => return Err(e)
}
}
}

// if no subdir was found, let's pop and delete
if !has_child_dir {
match rmdir(&rm_stack.pop().unwrap()) {
Ok(()) => (),
Err(ref e) if e.kind == io::FileNotFound => (),
Err(e) => return Err(e)
}
}
}
// Directory should now be empty
rmdir(path)

Ok(())
}

/// Changes the timestamps for a file's last modification and access time.
Expand Down Expand Up @@ -920,10 +968,36 @@ mod test {
check!(rmdir(dir));
})

iotest!(fn recursive_mkdir() {
let tmpdir = tmpdir();
let dir = tmpdir.join("d1/d2");
check!(mkdir_recursive(&dir, io::UserRWX));
assert!(dir.is_dir())
})

iotest!(fn recursive_mkdir_slash() {
check!(mkdir_recursive(&Path::new("/"), io::UserRWX));
})

// FIXME(#12795) depends on lstat to work on windows
#[cfg(not(windows))]
iotest!(fn recursive_rmdir() {
let tmpdir = tmpdir();
let d1 = tmpdir.join("d1");
let dt = d1.join("t");
let dtt = dt.join("t");
let d2 = tmpdir.join("d2");
let canary = d2.join("do_not_delete");
check!(mkdir_recursive(&dtt, io::UserRWX));
check!(mkdir_recursive(&d2, io::UserRWX));
check!(File::create(&canary).write(bytes!("foo")));
check!(symlink(&d2, &dt.join("d2")));
check!(rmdir_recursive(&d1));

assert!(!d1.is_dir());
assert!(canary.exists());
})

iotest!(fn unicode_path_is_dir() {
assert!(Path::new(".").is_dir());
assert!(!Path::new("test/stdtest/fs.rs").is_dir());
Expand Down