Skip to content

Cannot read unicode data from STDIN on Windows #5148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Tetralux opened this issue Apr 23, 2020 · 1 comment
Open

Cannot read unicode data from STDIN on Windows #5148

Tetralux opened this issue Apr 23, 2020 · 1 comment
Labels
os-windows standard library This issue involves writing Zig code for the standard library.
Milestone

Comments

@Tetralux
Copy link
Contributor

Tetralux commented Apr 23, 2020

Tracking issue for how, on Windows, just using ReadFile will completely corrupt UTF-16 data.

Instead, you have to use ReadConsoleW.

For example:

extern "kernel32" fn ReadConsoleW(handle: os.fd_t, buffer: [*]u16, len: os.windows.DWORD, read: *os.windows.DWORD, input_ctrl: ?*c_void) i32;
extern "kernel32" fn SetConsoleOutputCP(cp: os.windows.UINT) i32;

// ...

assert(SetConsoleOutputCP(65001) != 0); // display outputted utf8 correctly in the console
    
var stdin = io.getStdIn().handle;
var data: [256]u16 = undefined;
var read: u32 = undefined;
const ok = ReadConsoleW(stdin, &data, data.len, &read, null); // type in '안녕'
assert(ok != 0);

var utf8: [1024]u8 = undefined;
const utf8_len = try unicode.utf16leToUtf8(&utf8, data[0..read]);
const s = utf8[0..utf8_len];

print("'{}'\n", .{s});  // prints out '안녕
                        //            '

As noted by @fengb, this may mean Zig needs to have a special InStream for the console.

@Vexu Vexu added os-windows standard library This issue involves writing Zig code for the standard library. labels Apr 23, 2020
@Vexu Vexu added this to the 0.7.0 milestone May 6, 2020
@doongjohn
Copy link

doongjohn commented May 10, 2020

maybe we can solve this without using ReadConsoleW?
https://forum.nim-lang.org/t/5874#38898

edit: using ReadConsoleW seems cleaner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
os-windows standard library This issue involves writing Zig code for the standard library.
Projects
None yet
4 participants