start: Make nCmdShow better match how Windows sets it

[squeek502]

Partial follow up to #17763 (comment) (cc @e4m2)

nCmdShow is a parameter of WinMain that "Controls how the window is to be shown."

From the added comment:

    // This makes Zig mostly match the nCmdShow behavior of a C program with a WinMain symbol:
    // - With STARTF_USESHOWWINDOW set in STARTUPINFO.dwFlags of the CreateProcess call:
    //   - Compiled with subsystem:console -> nCmdShow is always SW_SHOWDEFAULT
    //   - Compiled with subsystem:windows -> nCmdShow is STARTUPINFO.wShowWindow from
    //     the parent CreateProcess call
    // - With STARTF_USESHOWWINDOW unset:
    //   - nCmdShow is always SW_SHOWDEFAULT
    //
    // TODO: Figure out how to check the subsystem and use it to match the above
    //       behavior, or intentionally avoid matching the above behavior with
    //       regards to the subsystem.

With regards to the TODO, it might make sense to just intentionally keep the current behavior: always forward wShowWindow if STARTF_USESHOWWINDOW is set regardless of the subsystem of the process, since it doesn't seem like a program compiled with the console subsystem would care about nCmdShow, or if they do, then I'm not sure why it makes sense to not set nCmdShow.

Tested the Windows behavior with the following setup:

---

// launch-show.c
#include <windows.h>
#include <stdio.h>
#include <tchar.h>
#include <shlwapi.h>

void _tmain( int argc, TCHAR *argv[] )
{
    STARTUPINFO si;
    PROCESS_INFORMATION pi;

    ZeroMemory( &si, sizeof(si) );
    si.cb = sizeof(si);
    ZeroMemory( &pi, sizeof(pi) );

    if( argc != 3 )
    {
        printf("Usage: %s [wShowWindow] [cmdline]\n", argv[0]);
        return;
    }

    si.wShowWindow = StrToInt(argv[1]);
    si.dwFlags |= STARTF_USESHOWWINDOW;

    // Start the child process. 
    if( !CreateProcess( NULL,   // No module name (use command line)
        argv[2],        // Command line
        NULL,           // Process handle not inheritable
        NULL,           // Thread handle not inheritable
        FALSE,          // Set handle inheritance to FALSE
        0,              // No creation flags
        NULL,           // Use parent's environment block
        NULL,           // Use parent's starting directory 
        &si,            // Pointer to STARTUPINFO structure
        &pi )           // Pointer to PROCESS_INFORMATION structure
    ) 
    {
        printf( "CreateProcess failed (%d).\n", GetLastError() );
        return;
    }

    // Wait until child process exits.
    WaitForSingleObject( pi.hProcess, INFINITE );

    // Close process and thread handles. 
    CloseHandle( pi.hProcess );
    CloseHandle( pi.hThread );
}

Compiled with:

zig build-exe launch-show.c -lc -lShlwapi

---

// winmain.c
#include <windows.h>
#include <stdio.h>
#include <winbase.h>

int APIENTRY WinMain(_In_ HINSTANCE hInstance,
                     _In_opt_ HINSTANCE hPrevInstance,
                     _In_ LPSTR    lpCmdLine,
                     _In_ int       nCmdShow)
{
    STARTUPINFO info;
    GetStartupInfo(&info);

    CHAR buf[256];
    wsprintf(buf, "WinMain: %d GetStartupInfo: %d", nCmdShow, info.wShowWindow);
    printf("%s\n", buf);
    MessageBoxA(NULL, buf, NULL, 0);
}

Compiled with:

zig build-exe winmain.c -lc -femit-bin=winmain-console.exe --subsystem console

and

zig build-exe winmain.c -lc -femit-bin=winmain-windows.exe --subsystem windows

---

> launch-show.exe 5 winmain-console.exe
WinMain: 10 GetStartupInfo: 5

(the STARTUPINFO returned from GetStartupInfo has the wShowWindow of the CreateProcess call, but nCmdShow passed to WinMain is SW_SHOWDEFAULT)

> launch-show.exe 5 winmain-windows.exe
<message box with the string "WinMain: 5 GetStartupInfo: 5">

If the si.dwFlags |= STARTF_USESHOWWINDOW; line in launch-show.exe is commented out, then nCmdShow in WinMain is always SW_SHOWDEFAULT.

[expikr]

Note: can also test via shell shortcuts like so:

which is basically a proxy to what you're doing programmatically, but still useful to test from the shell to verify the behavior indeed matches.

[squeek502]

Addressed the comments, this should be mergeable unless there are more.

I will have a follow-up PR for the subsystem stuff. Right now, std.builtin.subsystem.? == .Windows will always be true given how std.builtin.subsystem works which is fine (it just means we'll pass on the 'show window' value to processes compiled with the console subsystem).

[andrewrk]

Really appreciate those nice comments, thank you!