Skip to content

BufferedStream<File> doesn't maintain logical position in file with mixed reads/writes #17136

Closed
@ben0x539

Description

@ben0x539
Contributor

Here's an example:

use std::io::{println, File, Open, Truncate, Write, ReadWrite, BufferedStream};

fn main() {
    let path = Path::new("test");

    let mut file = File::open_mode(&path, Truncate, Write).unwrap();
    file.write_str("123456789").unwrap();
    drop(file);

    let file = File::open_mode(&path, Open, ReadWrite).unwrap();
    let mut bs = BufferedStream::new(file);
    println(bs.read_exact(2).unwrap().into_ascii().into_string().as_slice());
    bs.write_str("xyz").unwrap();
    bs.flush().unwrap();
    println(bs.read_to_string().unwrap().as_slice());
}

This prints

12
3456789

and afterwards the file contains the string 123456789xyz even though logically the write happens between the two reads and should have moved the file position over the 345 without them being returned from read calls instead of appending to the end (or overwriting the wrong bytes if a tiny buffer size is chosen). This makes the buffering leak through the abstraction where a mostly equivalent C program maintains the illusion:

#include <stdio.h>

int main(void) {
  char buf[10] = { 0 };
  FILE* f;

  f = fopen("test", "w");
  fwrite("123456789", 1, 9, f);
  fclose(f);

  f = fopen("test", "r+");
  fread(buf, 1, 2, f);
  puts(buf);
  fwrite("xyz", 1, 3, f);
  fread(buf, 1, 9, f);
  puts(buf);
  fclose(f);

  return 0;
}

printing

12
6789

and leaving the file to contain 12xyz6789. strace suggests glibc just seems to seek backwards (lseek(3, -7, SEEK_CUR)) before the write presumably for the length of the discarded read buffer or otherwise manually keeping track of the logical file position.

Activity

mahkoh

mahkoh commented on Jan 17, 2015

@mahkoh
Contributor

This has to be decided before 1.0.

pnkfelix

pnkfelix commented on Jan 22, 2015

@pnkfelix
Member

P-backcompat-libs, not 1.0. We already have a separate process for stabilizing the libraries and particular types within them, independent of the 1.0 relaese itself.

sfackler

sfackler commented on Jan 28, 2015

@sfackler
Member

I don't think that BufferedStream itself can handle this case. For many (most?) other Stream types, the input and output parts of the stream aren't connected in any way. For example, if I write to a TCP stream, I don't necessarily expect to be reading that back out.

I think the best solution here is probably a separate BufferedFile type that can properly keep everything in sync, while offering all of the extra methods like seek and truncate that you'd want.

brson

brson commented on Apr 24, 2015

@brson
Contributor

Renominating. Are we going to go with this as is?

pnkfelix

pnkfelix commented on Apr 30, 2015

@pnkfelix
Member

we need to decide what we were doing it. (We might mark BufferedStream unstable, or we might stick with status quo, or we might steal ideas from other languages...)

I-needs-decision, P-high.

aturon

aturon commented on Apr 30, 2015

@aturon
Member

8 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @ben0x539@alexcrichton@brson@pnkfelix@aturon

      Issue actions

        BufferedStream<File> doesn't maintain logical position in file with mixed reads/writes · Issue #17136 · rust-lang/rust