Cleanup code not executing in fixture

[zmitchell]

I'm having a problem with things executing out of order in a fixture. I'm using socat to create a pair of virtual serial ports so that I can test some code that interacts with serial devices. socat needs to be running the whole time the test is run, so I use a fixture to start it in a separate process. The issue is that if I don't wait at the beginning of the test the ports aren't open by the time the test runs. If I wait first, the test passes. In either case the socat process isn't terminated, even though I'm calling terminate on the process. Here is the code:

@pytest.fixture
def socat():
    proc = Process(target=socat_command)
    proc.start()
    yield None
    proc.terminate()
    proc.join()

def socat_command():
    os.system("socat pty,rawer,echo=0,link=/tmp/port1 pty,rawer,echo=0,link=/tmp/port2")

def test_can_connect_to_port(socat):
    # time.sleep(0.5)
    settings = {"port": "/tmp/port1", "baudrate": "115200"}
    lia = LockInAmplifier(port_settings=settings)
    trio.run(lia.connect)
    assert lia._ser is not None

Why isn't the process terminated at the end? Is there a better way to do this? I want a fresh set of ports for each test so that the input/output buffers are always clean. I feel like this is just something I'm doing wrong or misunderstanding, so hopefully you can point me in the right direction.

These are the versions pinned in my Pipfile.lock:

pytest==3.5.0
pydoc-markdown==2.0.2
pre-commit==1.8.2
hypothesis==3.55.6
flake8==3.5.0
black==18.4a2
attrs==17.4.0
trio==0.4.0
pyserial==3.4
logzero==1.5.0

I'm on macOS 10.13.3.

[pytestbot]

GitMate.io thinks possibly related issues are #517 (setup_* executed before all fixtures), #832 (some codes are not executed.), #3032 (Teardown for parametrized module scope fixtures is not executed in the teardown phase), #1907 (Cleanup MANIFEST.in), and #1895 (Incorrect finalize/cleanup order of fixtures when using request.getfixturevalue()).

[RonnyPfannschmidt]

i guess that you use a multiprocessing process

then what you do is completely broken and cant possibly work, as you terminate he python process,
your parent "pytest" process will take ownership of the socat process

if all you need is to run socat and terminate it, just use a subprocess.Popen object directly,
if you need more, you will need a more detailed setup

[zmitchell]

Yes, I'm using multiprocessing.Process to run socat in the background. If I understand you correctly, my problem is that I kill the Python process, not the socat process, so nothing is actually terminating the socat process.

Using Popen cleans up afterwards, which is an improvement, but I still have to add a time.sleep(...) at the beginning of the test to make it work properly. I'm guessing that's because socat takes some time to set itself up after it starts, but all Python knows is that socat has started.

[RonnyPfannschmidt]

i strongly suggest to do a socket connection based wait in the fixture after the popen call

[zmitchell]

Maybe there’s a misunderstanding, but I’m not using socat for sockets, I’m using it for serial ports.

[RonnyPfannschmidt]

thanks for pointing that out, i indeed missed the finer details of the cli args

[Zac-HD]

A try...finally clause might also help:

@pytest.fixture
def socat():
    proc = Process(target=socat_command)
    proc.start()
    try:
        yield None
    finally:
        proc.terminate()
        proc.join()