Windows: WindowsError in def _shutdown(self): in repository.py when running many commands

[null77]

Using a .23 build of stgit I could reproduce this every time just by running stg pull:

$ stg pull
Popping all applied patches ... done
Pulling from "origin"
Already up to date.
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\src\Python27\lib\atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "C:\src\Python27\lib\site-packages\stgit\lib\git\repository.py", line 122, in _shutdown
    os.kill(self._proc.pid(), signal.SIGTERM)
WindowsError: [Error 5] Access is denied
Error in sys.exitfunc:
Traceback (most recent call last):
  File "C:\src\Python27\lib\atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "C:\src\Python27\lib\site-packages\stgit\lib\git\repository.py", line 122, in _shutdown
    os.kill(self._proc.pid(), signal.SIGTERM)
WindowsError: [Error 5] Access is denied

I fixed this with the following update:

    def _shutdown(self):
        if self._proc:
            self._proc.stdin.close()
            try:
                os.kill(self._proc.pid(), signal.SIGTERM)
            except OSError:
                pass
            self._proc.wait()

I believe what's happening here is that the PID is no longer valid and we're trying to kill an already-dead process.

[jpgrayson]

Sorry for the delayed response. Thanks for reporting this issue and suggesting a repair.

It would be really helpful to know if this behavior still happens on the head of the master branch with Python 3.5+. If you'd be willing to try that out, it would be appreciated.

[null77]

Hey @jpgrayson , this issue happens to me on Windows with Python 3.9. It seems to be flaky. StackOverflow recommends using taskkill.exe as a fallback:

https://stackoverflow.com/questions/6688815/windowserror-error-5-access-is-denied

Doing what I proposed back in the description would at least silence the error.

[jpgrayson]

I appreciate the followup, @null77. I've gone ahead and applied the workaround you suggested.

I believe what is happening is that closing stdin of the child process causes that process to terminate. On non-Windows platforms, the process remains in a defunct/zombie state until wait() is called and thus the os.kill() call idempotently does nothing.

On Windows, however, the process model is different and it seems that the child process just ceases to exist such that os.kill() targets a non-existent process and thus fails as shown in this issue.

Another approach might have been to eliminate the self._proc.stdin.close() call, but without a Windows platform to test on, I went with the known working workaround.

[null77]

Thank you! I'll give the new version a shot.

[null77]

@jpgrayson oops, looks like somewhere in the shuffle this exception became

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\src\Python39\lib\site-packages\stgit\lib\git\repository.py", line 168, in _shutdown
    os.kill(self._proc.pid(), signal.SIGTERM)
PermissionError: [WinError 5] Access is denied

Originally I had posted it as a WindowsError. Now it seems the flake still happens because of the exception type being PermissionError.

[jpgrayson]

Thanks for the quick feedback, @null77.

Could you check the behavior with the self._proc.stdin.close() removed? I.e.:

    def _shutdown(self):
        if self._proc:
            os.kill(self._proc.pid(), signal.SIGTERM)
            self._proc.wait()

My theory is that the explicit close of stdin is causing this race and I don't know why explicitly closing stdin would be necessary anyway.

[jpgrayson]

I'm pretty confident that the latest changes (8206770 and 499ed60) resolve this. Closing.

[null77]

Thanks @jpgrayson ! Sorry I didn't follow-up earlier. I removed the stdin close and the error was no longer showing up.

[jpgrayson]

👍 Thank you for the follow-up @null77. Please file more issues if you see any more problems on Windows. Also, note discussion #152 which I just posted yesterday. I'd really like StGit to provide a good experience on Windows.