[CMake] Allow Python discovery from virtual environment

[hamidelmaazouz]

Currently the find_package(Python3) call binds with the system Python interpreter which does not necessarily have psutil needed to run integration tests (using LLVM LIT)

Installing psutil in the system Python environment is one workaround but absolutely not recommended. The hint added to CMake in this PR allows (the possibility of) discovery of Python when the virtual environment (via virtualenv or conda) is activated.

[teqdruid]

I don't understand this PR at all.

First, I compile CIRCT in a venv all the time and it picks up the Python executable just fine. (Unless I'm doing something wrong and just not noticing.)

Secondly, there is no CMakeUserPresets.json file in this PR.

Third, I don't understand how the changes to CMakeLists.txt would make any difference. You're setting a variable after find_package is called, so (if my understanding of CMake is correct) makes no difference to finding Python.

[hamidelmaazouz]

First, I compile CIRCT in a venv all the time and it picks up the Python executable just fine. (Unless I'm doing something wrong and just not noticing.)

Weird, it picks the system environment for me:

...
-- CIRCT Python bindings are disabled.
-- Found python at /usr/bin/python3.13
...

the system env is fine, just would not want to clutter it with packages. If it worked for you maybe the failure wouldn't be noticed if psutil had already been in the system ?

Secondly, there is no CMakeUserPresets.json file in this PR.

yeah I use preset files instead of typing commands, it's pretty standard practice even in llvm: https://github.com/llvm/llvm-project/blob/main/.gitignore#L27. That's why I'm proposing to gitignore it in the circt repo as well.

Third, I don't understand how the changes to CMakeLists.txt would make any difference. You're setting a variable after find_package is called, so (if my understanding of CMake is correct) makes no difference to finding Python.

Tried that before, for some reason the hint isn't taken into consideration when the variable is set before the find_package

I get the right environment discovered with this variable added after the find_package:

-- CIRCT Python bindings are disabled.
-- Found Python3: /home/hamidelmaazouz/miniforge3/envs/circt/bin/python3.11 (found version "3.11.12")

[teqdruid]

RE CMakeUserPresets.json: in my haste to review this, I didn't notice that it was in .gitignore. This change is perfectly fine and desirable.

[teqdruid]

-- Found Python3: /home/jodemme/_circt_venv313/bin/python3.13 (found suitable version "3.13.3", minimum required is "3.8") found components: Interpreter Development Development.Module Development.Embed

What's your PATH variable look like?

Everything I'm reading about Python3_FIND_VIRTUALENV says it needs to be set before find_python which is exactly what I would expect. I would encourage you to investigate further. To be clear, I have no objection to adding it but I'm deeply skeptical of its location. I suspect that you are doing something else which is causing cmake to pick up python from your venv and that Python3_FIND_VIRTUALENV has no effect.

[hamidelmaazouz]

-- Found Python3: /home/jodemme/_circt_venv313/bin/python3.13 (found suitable version "3.13.3", minimum required is "3.8") found components: Interpreter Development Development.Module Development.Embed

What's your PATH variable look like?

Everything I'm reading about Python3_FIND_VIRTUALENV says it needs to be set before find_python which is exactly what I would expect. I would encourage you to investigate further. To be clear, I have no objection to adding it but I'm deeply skeptical of its location. I suspect that you are doing something else which is causing cmake to pick up python from your venv and that Python3_FIND_VIRTUALENV has no effect.

I share the same view, I will dig deeper into it and circle back. Thanks !