Slow shell performance after running pyenv virtualenv-init

[cmcginty]

It looks like there is some issue on my system that is causing very poor runtime performance after running pyenv virtualenv-init. Related to #132

I think the issue is with pyenv verision-name. What steps can I take to debug this further?

[cmcginty]

This issue is caused by a 100ms delay added to every script executed with $()command substitution in the code. The entry point of pyenv sh-activate has number of sub-commands that get executed with this call. The call trace I'm seeing is:

pyenv sh-activate
-> pyenv-sh-activate                +100ms
  -> pyenv-version-name             +100ms
    -> pyenv-version-file           +100ms
    -> pyenv-version-file-read      +100ms
    -> pyenv-hooks                  +100ms
-> pyenv-virtualenv-prefix          +100ms
  -> pyenv-version-name             +100ms
    -> pyenv-version-file           +100ms
    -> pyenv-version-file-read      +100ms
    -> pyenv-hooks                  +100ms

All these calls add up to 1s+ of latency.

My theory is this is related to a virus scanner or carbonblack.

[cmcginty]

More progress. I can reproduce outside of pyenv now. 😄
I wrote a shell script that creates and calls another script. If the script has a shebang line in it, then the execution is horrendous.

#!/bin/bash
SUB_SCRIPT=$(mktemp)
if [[ $1 == "--bash" ]]; then
  echo "#!/bin/bash" > $SUB_SCRIPT
fi
echo "exit" >> $SUB_SCRIPT
chmod +x $SUB_SCRIPT
for X in $(seq 100); do
  $($SUB_SCRIPT)
done
rm $SUB_SCRIPT

$ time ./test.sh
./test.sh  0.05s user 0.08s system 37% cpu 0.341 total

$ time ./test.sh --bash
./test.sh --bash  0.24s user 0.23s system 3% cpu 11.869 total

Even on my unaffected system, the performance is measurably worse when shebang line exists.

$ time ./test.sh
./test.sh  0.07s user 0.08s system 94% cpu 0.156 total

$ time ./test.sh --bash
./test.sh --bash  0.15s user 0.16s system 87% cpu 0.351 total

[Honghe]

running into the same problem too, on Ubuntu 16.04.

[cmcginty]

@Honghe Does your laptop run CarbonBlack? That was the root cause of my performance issue.

[Honghe]

@cmcginty no running CarbonBlack ;(

[chriscarpenter12]

Came here looking for a solution too. Added this to my zsh config and saw terrible performance. On Ubuntu 18.04. Removing this eval "$(pyenv virtualenv-init -)" removes the slowness.

[Baishan]

Experiencing this^ too.

[jtherrmann]

Similar problem on Debian 9 with zsh. Removing eval "$(pyenv virtualenv-init -)" from ~/.zshrc fixes it.

[SakiiR]

any news ?

[jtherrmann]

What does eval "$(pyenv virtualenv-init -)" actually do? I just commented it out and have still been able to use pyenv with virtualenv with no problems.

[shawnohare]

What does eval "$(pyenv virtualenv-init -)" actually do? I just commented it out and have still been able to use pyenv with virtualenv with no problems.

It can automatically activate virtualenvs when a .python-version file is present.

[jtherrmann]

What does eval "$(pyenv virtualenv-init -)" actually do? I just commented it out and have still been able to use pyenv with virtualenv with no problems.

It can automatically activate virtualenvs when a .python-version file is present.

So it just removes the need to run source venv/bin/activate?

[mayanksuman]

Yes, echo $(pyenv virtualenv-init -) makes every command slow. This command run following set of command while starting the shell.

export PATH="/home/mayank/.pyenv/plugins/pyenv-virtualenv/shims:${PATH}";
export PYENV_VIRTUALENV_INIT=1;
_pyenv_virtualenv_hook() {
  local ret=$?
  if [ -n "$VIRTUAL_ENV" ]; then
    eval "$(pyenv sh-activate --quiet || pyenv sh-deactivate --quiet || true)" || true
  else
    eval "$(pyenv sh-activate --quiet || true)" || true
  fi
  return $ret
};
typeset -g -a precmd_functions
if [[ -z $precmd_functions[(r)_pyenv_virtualenv_hook] ]]; then
  precmd_functions=(_pyenv_virtualenv_hook $precmd_functions);
fi

Hence, other than adding virtualenv in $PATH, it appends a hook to be run before every command. The only purpose of the hook is to activate virtualenvs when a .python-version file is present.

[laggardkernel]

Solution

Warning: the solution is far from perfect, it breaks command pyenv activate/deactivate. Use it only if the slowness really bothers you.

Quick Solution

1. Unload hook func _pyenv_virtualenv_hook
2. Never use pyenv activate/deactivate, cause they're broken without the above hook. Stick with pyenv shell env_name, pyenv shell --unset instead.

For ZSH and Bash

# Init pyenv-virtualenv, but 
# unload precmd hook _pyenv_virtualenv_hook
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

# Warning: unloading the following hook breaks command
# `pyenv activate/deactivate`. Please switch to
# `pyenv shell env_name`, `pyenv shell --unset` instead.
if [[ -n $ZSH_VERSION ]]; then
  autoload -Uz add-zsh-hook
  add-zsh-hook -D precmd _pyenv_virtualenv_hook
fi
if [[ -n $BASH_VERSION ]]; then
  PROMPT_COMMAND="${PROMPT_COMMAND/_pyenv_virtualenv_hook;/}"
fi

[jbojar]

Slow shell performance with pyenv-virtualenv enabled can be caused by some antivirus software, in my case it was ESET. I compared two macbooks with same MacOS version (Catalina), one with and one without ESET. On the system with antivirus installed shell was very sluggish, on the other system its responsiveness was normal.
After upgrade to MacOS Big Sur ESET antivirus was removed (because of same incompatibility) and the responsiveness of comamnd prompt returned back to normal.

[ludocode]

I'm finding this slow as well. I'm on latest Arch Linux, nothing special about my configuration, and yet:

$ time _pyenv_virtualenv_hook

real	0m0.220s
user	0m0.160s
sys	0m0.070s

This is on the high end. For me it varies quite a bit, anywhere from 80 ms to 225 ms. This is way too much delay to add to every shell invocation.

The solution posted above is useful, but it does mean that activation is no longer automatic. My scripts check that PYENV_VERSION is correct to force me to activate manually which is a useable workaround.

[paprikman]

I am getting this as well on pyenv 1.2.25-2482-g9a42b48d and its previous versions. Having virtualenv-init in my .bashrc results in sensible delay even for just pressing enter with empty string in bash on ubuntu 18.04. I do not have zsh, fish, elvish or anything else installed on top, just plain bash. Without virtualenv-init the next line in prompt appears instantly.

[native-api]

To find out what takes time, we need to get a debug trace of what's running while you perform those actions and how long it takes.

For the first point

export PS4='+(${BASH_SOURCE}:${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'
set -x
<reproduce the problem>
set +x

For the second point, using $SECONDS in PS4 may help. (I'm not sure what it does, but it seems to expand to the time since last shell prompt or invocation.)
It that var doesn't help, you'll need to intersperse the code that runs when you reproduce the problem with time calls and use those to localize the time hog.

[raph-rc]

@raphaelchristin instead of chpwd use precwd. I'm on macos and it works perfectly fine for me.

❯ /bin/zsh --version
zsh 5.8.1 (x86_64-apple-darwin22.0)

works perfect, thank you so much!

[furudean]

For anyone using fish and looking for an equivalent to the zsh workaround, this seems to work for me:

status --is-interactive; and pyenv virtualenv-init - | sed 's/--on-event fish_prompt/--on-variable PWD/g' | source

Here's an alternative

if status is-interactive;
    pyenv init - | source
    pyenv virtualenv-init - | sed 's/--on-event fish_prompt/--on-event fish_preexec/g' | source
end

fish_preexec runs before the command is processed. This moves the toil to before the command executes. This lets you type new commands without slowing you down

[reegnz]

Well, I tried fixing the poor perf issue. Seems like the maintainers are not interested in fixing performance unless it's backward-compatible. So the hacks outside the project remain. 🤷

[furudean]

@reegnz I think what is suggested here is to fix the core problem--which is that pyenv is slow to run and isn't very clever with how it runs code--and not add band-aid fixes that break functionality for people. See #451 (comment)

[dsoprea]

Seconded, since not everyone uses zsh and most of the conversation has been dominated by zsh-specific solutions.

[w-biggs]

For anyone using fish and looking for an equivalent to the zsh workaround, this seems to work for me:

status --is-interactive; and pyenv virtualenv-init - | sed 's/--on-event fish_prompt/--on-variable PWD/g' | source

I couldn't get this to work on my setup, but this works on fish version 3.6.1 and pyenv 2.3.17:

if status is-interactive;
    pyenv init - --no-rehash | source
    pyenv virtualenv-init - | sed 's/--on-event fish_prompt/--on-event fish_preexec/g' | source
end

First, init pyenv with the --no-rehash option, this seems to speed things up a a bunch. (I'm not sure if there is any caveat to this!)

Secondly, I opted for the fish_preexec hook to run before the command is processed. I couldn't get the $PWD behavior working, but this feels quite a bit faster since you don't have to wait for the prompt to appear until you can type. This is similar to the actual behavior of the zsh workaround.

Not sure if I'm doing something wrong but this seems to still result in any command -- eg echo (gdate +%s%3N) -- taking about 1.4 seconds to execute on my machine. Commenting out only the virtualenv-init line results in it being near-instantaneous again.

For now, I think I'll just do things manually (adding pyenv activate -q [name of venv] to config.fish), but 1.4s per command seems a bit exorbitant.

[sunnstix]

Is there a similar bash workaround that I can use?

[hassec]

FWIW, I've tried using

eval "$(pyenv virtualenv-init - | sed s/precmd/precwd/g)"

and it does speed up the prompt, but it does seem to also lead to to PYENV_VIRTUAL_ENV and VIRTUAL_ENV not getting set, which can cause other tools to behave unexpectedly.

E.g. poetry will not recognize that a virtual environment is activated.

[yogiyojyno96]

eval "$(pyenv virtualenv-init - | sed s/precmd/chpwd/g)"

When using virtualenv-init in pyenv, by default, _pyenv_virtualenv_hook runs every time a shell prompt appears. To avoid this, I tried to change to use chpwd instead of precmd hook. This improves the response speed of the shell because the _pyenv_virtualenv_hook runs only upon directory changes.

However, if this method was applied, the virtual environment was not activated immediately because the _pyenv_virtualenv_hook did not run when zsh was first run. As a workaround, I modified the hook with the sed command and added the code to run _pyenv_virtualenv_hook once. This enabled the virtual environment to be activated normally at the start of zsh. I hope this information is helpful!

eval "$(pyenv virtualenv-init - | sed s/precmd/chpwd/g)"
_pyenv_virtualenv_hook

[owebeeone]

Just another data point. I was seeing a ~1.5sec delay on every prompt. Had to remove the hook.

[daniel-centore-at-datavant]

@yogiyojyno96 Thanks, your solution worked great!

I've modified it for the fish shell like this ~/.config/fish/config.fish:

pyenv virtualenv-init - | pyenv virtualenv-init - | sed s/"--on-event fish_prompt"/"--on-variable PWD"/g | source
_pyenv_virtualenv_hook

[FanchenBao]

In my case, the root problem is that pytenv sh-active is slow and it is executed each time _pyenv_virtualenv_hook is called. And since the hook is called each time we issue a command (or switch directory, if we change precmd to chpwd), it makes all actions on the shell sluggish.

To speed up the shell, we need to make the execution of pytenv sh-active more selective. Below is my work-around on this issue.

# Comment out the default command
# eval "$(pyenv virtualenv-init -)"

export PATH="/usr/local/Cellar/pyenv-virtualenv/1.2.1/shims:${PATH}";
export PYENV_VIRTUALENV_INIT=1;

TEMP_PYENV_VIRTUALENV_PROJECT_DIR=$(mktemp /tmp/pyenv_virtualenv_project_dir_$(date +"%Y-%m-%d_%T"))
 _pyenv_virtualenv_hook() {
  local ret=$?
  if [ -f $TEMP_PYENV_VIRTUALENV_PROJECT_DIR ]; then
    project_dir=$(cat $TEMP_PYENV_VIRTUALENV_PROJECT_DIR)
  else
    TEMP_PYENV_VIRTUALENV_PROJECT_DIR=$(mktemp /tmp/pyenv_virtualenv_project_dir_$(date +"%Y-%m-%d_%T"))
    project_dir=""
  fi
  if [[ $project_dir == "" ]]; then
    if [ -f .python-version ] || [ -d venv ]; then
      echo $PWD > $TEMP_PYENV_VIRTUALENV_PROJECT_DIR
      eval "$(pyenv sh-activate --quiet || true)" || . venv/bin/activate 2> /dev/null || true
    fi
  elif [[ ! $PWD =~ $project_dir ]]; then
    echo > $TEMP_PYENV_VIRTUALENV_PROJECT_DIR
    eval "$(pyenv sh-deactivate --quiet || true)" || deactivate 2> /dev/null ||true
  fi
  return $ret
};
typeset -g -a precmd_functions
if [[ -z $precmd_functions[(r)_pyenv_virtualenv_hook] ]]; then
  precmd_functions=(_pyenv_virtualenv_hook $precmd_functions);
fi

function shellExit {
    rm "$TEMP_PYENV_VIRTUALENV_PROJECT_DIR" 2> /dev/nul
}
trap shellExit EXIT

This is just a modification of the output of pyenv virtualenv-init -. The logic is changed regarding where and when pytenv sh-active is called. Briefly put, we only activate virtualenv if we see .python-version for the first time. We only deactivate if the current directory does not contain the project root. We use a temp file to keep track of the project root directory.

This method suits my need (i.e. always go to the project root first), but it is not a universal solution. For instance, if we initially cd to a folder deeper than the project root, the virtualenv will NOT be activated. Also, if we cd from one project root to another, the second project's virtualenv will not be sourced. To source the second, one must execute some command under that root, such as ls.

On the bright side, this work-around also supports automatically sourcing virtual environment created from python -m venv venv.

[ardeidae]

I am confused. I removed the eval "$(pyenv virtualenv-init -)" line from my .zshrc and my virtual env is still automatically activated when a .python-version file is found. So I don't understand what this line is for. Does anyone know the answer?

[hassec]

@ardeidae with the eval command removed cd into the directory that you mention.
Then do

echo $VIRTUAL_ENV
pyenv activate $(/bin/cat .python-version)
echo $VIRTUAL_ENV

I expect the first echo won't show anything and the 2nd will print the virtualenv name.
What you are seeing is likely that your prompt checks for .python-version or pyenv and thus already shows the name of the virutal env without it being activated.

[ardeidae]

@hassec

$ mkdir test && cd test

$ pyenv versions
system
3.11.0
* 3.12.1 (set by /Users/me/.pyenv/version)

$ pyenv virtualenv 3.11.0 test

$ pyenv local test

$ cat .python-version
test

$ echo $VIRTUAL_ENV


$ pyenv versions
system
3.11.0
3.11.0/envs/test
3.12.1
* test --> /Users/me/.pyenv/versions/3.11.0/envs/test (set by /Users/me/test/.python-version)

$ pyenv activate $(/bin/cat .python-version)

(test) $ echo $VIRTUAL_ENV
/Users/me/.pyenv/versions/3.11.0/envs/test

The only benefit I see is the "test" string added in the prompt, but without the eval, it seems the used virtual environment is the expected one.

If I use pyenv activate test, I also see this string displayed in the prompt.

[hassec]

I don't think I know all the details around this issue, but I believe that the virtualenv is not actually activated. When you call python it picks the right one because pyenv resolves it with its shim mechanism.

But it's technically not equivalent to actually having activated the env as shown by the absence of $VIRTUAL_ENV.

If e.g. starting neovim in that directory, it won't pick the python from within that environment. And I anticipate there are similar other corner cases, but this issue isn't the right place to discuss this further, I think.

[ardeidae]

@hassec
Using pip freeze to see what is installed returns the correct list of packages.
So for me, this uses the right virtual environment without using eval.

[13steinj]

In my case, the root problem is that pytenv sh-active is slow and it is executed each time _pyenv_virtualenv_hook is called. And since the hook is called each time we issue a command (or switch directory, if we change precmd to chpwd), it makes all actions on the shell sluggish.

To speed up the shell, we need to make the execution of pytenv sh-activ[at]e more selective. Below is my work-around on this issue.

Emphasis mine, just doing this blind obscures the problem, not solves it. The solution is to actually improve the performance of pyenv-sh-activate (or rather the hook, that currently calls it). Most of the negative performance appears to be caused by recursive call into the pyenv shell function / command to call other pyenv[-virtualenv] scripts as well as pyenv libexec utilities.

So, for the sake of a fresh look at this investigative process:

If we can assume that we know what's a libexec and what isn't, and that we don't have to depend on the idea of someone else overriding pyenv or this plugin's behavior, a fix can be seen as follows:

• Remove unnecessary string -> value evaluation (rely on exit codes)
• determine the script bindir at the top of the script, use it directly.
• if no venv names are provided on the command line, no need to check the python prefix twice. ref:

  pyenv-virtualenv/bin/pyenv-sh-activate

  Lines 111 to 117 in f8469a1

  	if ! pyenv-virtualenv-prefix "${venv}" 1>/dev/null 2>&1; then
  	# fallback to virtualenv of current version
  	[ -n "${current_versions}" ] || get_current_versions
  	new_venv="${current_versions%/envs/*}/envs/${venv}"
  	if pyenv-virtualenv-prefix "${new_venv}" 1>/dev/null 2>&1; then
  	venv="${new_venv}"
  	versions[0]="${new_venv}"

  • If getting 3.12.2 as a local/global/shell version, we won't find a venv at 3.12.2/envs/3.12.2.
  • If getting some_venv_name as a local/global/shell version, we won't find a venv at some_venv_name/envs/some_venv_name
  • This shaves off 20-40 millis.

The first check still takes 20-40 millis. Can we do any better? Littering the scripts with date +%s%N | cut -b1-13 1>&2 (and timing this under zsh to take ~2.5ms, so, count of output x 2.5 => time spent on the date calls themselves) shows me this pattern:

Which tells me that the actual prefix finding script isn't a large deal. Digging deeper, get_current_versions is 40-60ms (incredibly variable), each call to the prefix versions script is ~ 20ms. Not much point in optimizing out the other call then. Can we do anything about get_current_versions?

Logically speaking, the answer is yes-- it's effectively the result of https://github.com/pyenv/pyenv/blob/5b4d5a32d343dcae5e7b3f1a09850312f89ba868/libexec/pyenv-version-name. The current versions (assuming no other plugins) can change under the result of the following commands:

• a changed directory
• the relevant .python_version file changing (if done manually or directly by some plugin, screw that)
• pyenv shell

So, the ideal would be

• don't check venv prefixes twice unnecessarily
  • have the singular call be a bash function in the sh-activate script rather than it's own script? sourcing is relatively fast by my benchmarks, so might want to move it to libexec and source that in the bin/ script.
• call the scripts directly rather than via indirection
• don't go back and forth between strings and values unnecessarily
• only read the .python_version file if you have to. Which means
  • if PYENV_VERSION is set, skip calling get_current_versions spawning subshells (that info is already in the env var).
  • cache the result of get_current_versions somehow. Maybe change the hook to evaluate on chpwd, get the versions, cache the dir + versions tuple,
    pass the versions in to sh-activate, cache-miss on changed directory (chpwd), set a separate hook such that every time pyenv global/local are called, the versions cache gets cleared?

I'll try to implement this ideal over the next weekend (only focusing on zsh and by extension that the changes are not magic, bash , sorry) on a branch; and this should shave off the majority (>90ms/120ms per call) by my estimate.

[kaddkaka]

Any reactions from devs?

[native-api]

Any reactions from devs?

If you hover the reaction icon under the last post, you'll see mine in the list. We'll happily accept a PR.

To do anything ourselves, we need a way to reproduce the issue or a debug trace with timestamps from someone who experiences it.

[kaddkaka]

Thanks, I understand.

My situation: just doing the pyenv virtualen init in my bashrc on Ubuntu 24.04 caused my prompt to appear very slow in wezterm terminal. To notice I held my enter key down for 5 seconds in the terminal and got several empty lines between each prompt.

[fredcallaway]

I'm getting a ~200ms delay at each prompt. It is all coming from pyenv sh-activate.

❯ time pyenv sh-activate
pyenv-virtualenv: version `3.11.11' is not a virtualenv
false
pyenv sh-activate  0.05s user 0.11s system 81% cpu 0.200 total

~
❯ time pyenv sh-activate
pyenv-virtualenv: version `3.11.11' is not a virtualenv
false
pyenv sh-activate  0.05s user 0.11s system 80% cpu 0.196 total

~
❯ time pyenv sh-activate
pyenv-virtualenv: version `3.11.11' is not a virtualenv
false
pyenv sh-activate  0.05s user 0.11s system 80% cpu 0.199 total