Skip to content

Tab completion for pip config suggests invalid options. #13133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 task done
hongyi-zhao opened this issue Dec 29, 2024 · 5 comments · Fixed by #13140
Closed
1 task done

Tab completion for pip config suggests invalid options. #13133

hongyi-zhao opened this issue Dec 29, 2024 · 5 comments · Fixed by #13140
Labels
C: autocomplete Autocompletion in shells (pip completion) type: bug A confirmed bug or unintended behavior

Comments

@hongyi-zhao
Copy link

hongyi-zhao commented Dec 29, 2024

Description

When using tab completion with pip config -[TAB], it suggests general pip options like --cache-dir, --cert, etc., as shown below:

(miniforge3-24.11.0-1) werner@x13dai-t:~$ pip config -
--cache-dir=                 --global                     --no-cache-dir=              --require-venv               --user
--cert=                      -h                           --no-color                   --require-virtualenv         -v
--client-cert=               --help                       --no-input                   --retries=                   -V
--debug                      --isolated                   --no-python-version-warning  --site                       --verbose
--default-timeout=           --keyring-provider=          --proxy=                     --timeout=                   --version
--disable-pip-version-check  --local-log=                 --python=                    --trusted-host=              
--editor=                    --log=                       -q                           --use-deprecated=            
--exists-action=             --log-file=                  --quiet                      --use-feature=   

However, pip config requires an action (debug, edit, get, list, set, unset) first and these suggestions are not given.

Expected behavior

Tab completion should show valid actions (debug, edit, get, list, set, unset) or their relevant options.

pip version

(miniforge3-24.11.0-1) werner@x13dai-t:~$ pip --version pip 24.3.1 from /home/werner/.pyenv/versions/miniforge3-24.11.0-1/lib/python3.12/site-packages/pip (python 3.12)

Python version

(miniforge3-24.11.0-1) werner@x13dai-t:~$ python --version
Python 3.12.8

OS

(miniforge3-24.11.0-1) werner@x13dai-t:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.4 LTS
Release: 22.04
Codename: jammy

How to Reproduce

See above.

Output

See above.

Code of Conduct

@hongyi-zhao hongyi-zhao added S: needs triage Issues/PRs that need to be triaged type: bug A confirmed bug or unintended behavior labels Dec 29, 2024
@matthewhughes934
Copy link
Contributor

matthewhughes934 commented Jan 2, 2025

It looks like pip's completion isn't aware of arguments that aren't flags (i.e. not starting with - or --). At a glance this also impacts the cache and index commands.

I do see that each of those commands look to have a fairly common setup: they define a dict of handlers mapping names to methods taking the same argument types, check that args[0] is one of those names (and not a flag like --global). Maybe the common bits (at least the dictionary of names -> functions) could be moved to some base class that the completion command could then access.

EDIT: there's also discussion #4659 so maybe if pip is moving to another library for arg parsing it's easier to add that functionality then, rather than introduce it in the current setup

@ichard26
Copy link
Member

ichard26 commented Jan 3, 2025

there's also discussion #4659 so maybe if pip is moving to another library for arg parsing it's easier to add that functionality then, rather than introduce it in the current setup

Unless you're signing up as the volunteer who will lead a migration to a new argument parsing framework, I can assure you that ain't going to be happening any time soon 🙂

matthewhughes934 added a commit to matthewhughes934/pip that referenced this issue Jan 3, 2025
For commands like `pip cache` with sub-actions like `remove`, so that
e.g. `pip cache re<TAB>` completes to `pip cache remove`.

All the existing commands that used such sub-actions followed the same
approach for: using a dictionary of names to methods to run, so the
implementation is just  teaching the `Command` object about this mapping
and using it in the autocompletion function.

There's no handling for the position of the argument, so e.g. `pip cache
re<TAB>` and `pip  cache --user re<TAB>` will both complete the final
word to `remove`. This is mostly because it was simpler to implement like
this, but also I think due to how `optparse` works such invocations are
valid, e.g. `pip config --user set global.timeout 60`. Similarly,
there's no duplication handling so `pip cache remove re<TAB>` will also
complete.

This is a feature that may be simpler to implement, or just work out of
the box, with some argument parsing libraries, but moving to another
such library looks to be quite a bit of work (see discussion[1]).

I also took the opportunity to tighten some typing: dropping some use of
`Any`

Link: pypa#4659 [1]
Fixes: pypa#13133
matthewhughes934 added a commit to matthewhughes934/pip that referenced this issue Jan 3, 2025
For commands like `pip cache` with sub-actions like `remove`, so that
e.g. `pip cache re<TAB>` completes to `pip cache remove`.

All the existing commands that used such sub-actions followed the same
approach for: using a dictionary of names to methods to run, so the
implementation is just  teaching the `Command` object about this mapping
and using it in the autocompletion function.

There's no handling for the position of the argument, so e.g. `pip cache
re<TAB>` and `pip  cache --user re<TAB>` will both complete the final
word to `remove`. This is mostly because it was simpler to implement like
this, but also I think due to how `optparse` works such invocations are
valid, e.g. `pip config --user set global.timeout 60`. Similarly,
there's no duplication handling so `pip cache remove re<TAB>` will also
complete.

This is a feature that may be simpler to implement, or just work out of
the box, with some argument parsing libraries, but moving to another
such library looks to be quite a bit of work (see discussion[1]).

I also took the opportunity to tighten some typing: dropping some use of
`Any`

Link: pypa#4659 [1]
Fixes: pypa#13133
matthewhughes934 added a commit to matthewhughes934/pip that referenced this issue Jan 3, 2025
For commands like `pip cache` with sub-actions like `remove`, so that
e.g. `pip cache re<TAB>` completes to `pip cache remove`.

All the existing commands that used such sub-actions followed the same
approach for: using a dictionary of names to methods to run, so the
implementation is just  teaching the `Command` object about this mapping
and using it in the autocompletion function.

There's no handling for the position of the argument, so e.g. `pip cache
re<TAB>` and `pip  cache --user re<TAB>` will both complete the final
word to `remove`. This is mostly because it was simpler to implement like
this, but also I think due to how `optparse` works such invocations are
valid, e.g. `pip config --user set global.timeout 60`. Similarly,
there's no duplication handling so `pip cache remove re<TAB>` will also
complete.

This is a feature that may be simpler to implement, or just work out of
the box, with some argument parsing libraries, but moving to another
such library looks to be quite a bit of work (see discussion[1]).

I also took the opportunity to tighten some typing: dropping some use of
`Any`

Link: pypa#4659 [1]
Fixes: pypa#13133
@matthewhughes934
Copy link
Contributor

Unless you're signing up as the volunteer who will lead a migration to a new argument parsing framework, I can assure you that ain't going to be happening any time soon 🙂

Tempting, though after a quick look I couldn't find a way to do this piece-wise (i.e. without having to rewrite it all at once) so just tried a simple change for this issue instead: #13140

@ichard26 ichard26 added C: autocomplete Autocompletion in shells (pip completion) and removed S: needs triage Issues/PRs that need to be triaged labels Jan 4, 2025
matthewhughes934 added a commit to matthewhughes934/pip that referenced this issue Apr 23, 2025
For commands like `pip cache` with sub-actions like `remove`, so that
e.g. `pip cache re<TAB>` completes to `pip cache remove`.

All the existing commands that used such sub-actions followed the same
approach for: using a dictionary of names to methods to run, so the
implementation is just  teaching the `Command` object about this mapping
and using it in the autocompletion function.

There's no handling for the position of the argument, so e.g. `pip cache
re<TAB>` and `pip  cache --user re<TAB>` will both complete the final
word to `remove`. This is mostly because it was simpler to implement like
this, but also I think due to how `optparse` works such invocations are
valid, e.g. `pip config --user set global.timeout 60`. Similarly,
there's no duplication handling so `pip cache remove re<TAB>` will also
complete.

This is a feature that may be simpler to implement, or just work out of
the box, with some argument parsing libraries, but moving to another
such library looks to be quite a bit of work (see discussion[1]).

I also took the opportunity to tighten some typing: dropping some use of
`Any`

Link: pypa#4659 [1]
Fixes: pypa#13133
matthewhughes934 added a commit to matthewhughes934/pip that referenced this issue Apr 23, 2025
For commands like `pip cache` with sub-commands like `remove`, so that
e.g. `pip cache re<TAB>` completes to `pip cache remove`.

All the existing commands that used such sub-commands followed the same
approach: using a dictionary of names to methods to run, so the
implementation is just  teaching the `Command` object about this mapping
and using it in the autocompletion function.

There's no handling for the position of the argument, so e.g. `pip cache
re<TAB>` and `pip  cache --user re<TAB>` will both complete the final
word to `remove`. This is mostly because it was simpler to implement like
this, but also I think due to how `optparse` works such invocations are
valid, e.g. `pip config --user set global.timeout 60`.

This is a feature that may be simpler to implement, or just work out of
the box, with some argument parsing libraries, but moving to another
such library looks to be quite a bit of work (see discussion[1]).

I also took the opportunity to tighten some typing: dropping some use of
`Any`

Link: pypa#4659 [1]
Fixes: pypa#13133
matthewhughes934 added a commit to matthewhughes934/pip that referenced this issue Apr 24, 2025
For commands like `pip cache` with sub-commands like `remove`, so that
e.g. `pip cache re<TAB>` completes to `pip cache remove`.

All the existing commands that used such sub-commands followed the same
approach: using a dictionary of names to methods to run, so the
implementation is just  teaching the `Command` object about this mapping
and using it in the autocompletion function.

There's no handling for the position of the argument, so e.g. `pip cache
re<TAB>` and `pip  cache --user re<TAB>` will both complete the final
word to `remove`. This is mostly because it was simpler to implement like
this, but also I think due to how `optparse` works such invocations are
valid, e.g. `pip config --user set global.timeout 60`.

This is a feature that may be simpler to implement, or just work out of
the box, with some argument parsing libraries, but moving to another
such library looks to be quite a bit of work (see discussion[1]).

I also took the opportunity to tighten some typing: dropping some use of
`Any`

Link: pypa#4659 [1]
Fixes: pypa#13133
@hongyi-zhao
Copy link
Author

hongyi-zhao commented May 15, 2025

I tried the PRs mentioned above, and below are some of my comments:

  1. The issue thas been fixed, as shown below:
(datasci) werner@x13dai-t:~$ pip config <tab>
--cache-dir=                 --exists-action=             --log=                       --require-venv               --use-deprecated=
--cert=                      get                          --log-file=                  --require-virtualenv         --use-feature=
--client-cert=               --global                     --no-cache-dir=              --resume-retries=            --user
--debug                      -h                           --no-color                   --retries=                   -v
debug                        --help                       --no-input                   set                          -V
--default-timeout=           --isolated                   --proxy=                     --site                       --verbose
--disable-pip-version-check  --keyring-provider=          --python=                    --timeout=                   --version
edit                         list                         -q                           --trusted-host=              
--editor=                    --local-log=                 --quiet                      unset 
  1. For other subcommands, say, install, show, etc., only the pip sub_command -<tab> works, pip sub_command <tab> won't give the expected options. I don't know if this is the expected behavior. See below for more details:
(datasci) werner@x13dai-t:~$ pip show -<tab>
--cache-dir=                 -f                           --log=                       -q                           --trusted-host=
--cert=                      --files                      --log-file=                  --quiet                      --use-deprecated=
--client-cert=               -h                           --no-cache-dir=              --require-venv               --use-feature=
--debug                      --help                       --no-color                   --require-virtualenv         -v
--default-timeout=           --isolated                   --no-input                   --resume-retries=            -V
--disable-pip-version-check  --keyring-provider=          --proxy=                     --retries=                   --verbose
--exists-action=             --local-log=                 --python=                    --timeout=                   --version

(datasci) werner@x13dai-t:~$ pip show <tab>
Display all 687 possibilities? (y or n)y
abipy                          fasteners                      mccabe                         pure-eval                      smmap
absl-py                        fastjsonschema                 mdanalysis                     py3dmol                        sniffio
accessible-pygments            fastlite                       mda-xdrlib                     py4vasp                        snowballstemmer
aenum                          fdint                          mdit-py-plugins                pyarrow                        socksio
aiofiles                       feedparser                     mdtex2html                     pyaudio                        sortedcontainers
aiohappyeyeballs               ffmpy                          mdurl                          pyautogen                      sounddevice
aiohttp                        filelock                       memory-profiler                pybind11                       soupsieve
aioitertools                   filetype                       mendeleev                      pybind11-global                spacy
aiosignal                      fireworks                      mergedeep                      pybtex                         spacy-legacy
aiosqlite                      flake8                         meshcut                        pycairo                        spacy-loggers
alabaster                      flaml                          mistune                        pycifrw                        spark-parser
albumentations                 flask                          mkdocs                         pycodestyle                    sparse
altair                         flask-caching                  mkdocs-autorefs                pycparser                      spglib
amcheck                        flask-paginate                 mkdocs-material                pycryptodomex                  sphinx
amset                          flatbuffers                    mkdocs-material-extensions     pycurl                         sphinx-argparse
annotated-types                flatlatex                      mkdocstrings                   pydantic                       sphinx-autodoc-typehints
anthropic                      flatten-dict                   mkl                            pydantic-core                  sphinx-basic-ng
anyio                          flexcache                      ml-dtypes                      pydantic-settings              sphinx-book-theme
apeye                          flexparser                     mmtf-python                    pydash                         sphinxcontrib-applehelp
apeye-core                     flufl-lock                     mongogrant                     pydata-sphinx-theme            sphinxcontrib-devhelp
appdirs                        fonttools                      mongomock                      pydeck                         sphinxcontrib-htmlhelp
apscheduler                    fqdn                           monotonic                      pydispatcher                   sphinxcontrib-jquery
apsw                           frozendict                     monty                          pydot                          sphinxcontrib-jsmath
apswutils                      frozenlist                     more-itertools                 pydub                          sphinxcontrib-napoleon
argon2-cffi                    fsspec                         moto                           pyerfa                         sphinxcontrib-qthelp
argon2-cffi-bindings           furo                           moyopy                         pyfftw                         sphinxcontrib-serializinghtml
arrow                          future                         mp-api                         pyfiglet                       sphinx-copybutton
arxiv                          fuzzysearch                    mpcontribs-client              pyflakes                       sphinx-design
ase                            galore                         mpmath                         pygments                       sphinx-jinja2-compat
astropy                        gast                           mp-pyrho                       pygobject                      sphinx-markdown-tables
astropy-iers-data              gb-code                        mrcfile                        pyhull                         sphinx-prompt
asttokens                      ghp-import                     msgpack                        pyisemail                      sphinx-rtd-theme
astunparse                     gitdb                          multidict                      pyjwt                          sphinx-tabs
async-lru                      gitpython                      multiprocess                   pylatexenc                     sphinx-toolbox
async-timeout                  google-pasta                   munch                          pymatgen                       sqlalchemy
atomate                        gradio                         murmurhash                     pymatgen-analysis-defects      srsly
atomate2                       gradio-client                  mutagen                        pymatgen-analysis-diffusion    sshtunnel
atpublic                       greenlet                       mypy-extensions                pymatgen-db                    stack-data
attrdict                       griddataformats                myst-nb                        pymatgen-io-validation         starlette
attrs                          grpcio                         myst-parser                    pymdown-extensions             starsessions
autodoc-pydantic               gunicorn                       namex                          pymongo                        stmol
autodocsumm                    h11                            narwhals                       pympler                        streamlit
autoflake                      h5py                           natsort                        pymupdf                        sumo
autopep8                       hetbuilder                     nbclassic                      pymupdfb                       supervisor
babel                          hiphive                        nbclient                       pynacl                         surfflow
backcall                       htflow-utils                   nbconvert                      pynput                         swagger-spec-validator
backoff                        html5lib                       nbdime                         pyparsing                      symfc
bcrypt                         htmlmin                        nbformat                       pypdf                          sympy
beautifulsoup4                 httpcore                       nbmake                         pypdf2                         tabulate
black                          httptools                      nbstripout                     pypdfium2                      tbb
bleach                         httpx                          nest-asyncio                   pyperclip                      tblite
blessed                        httpx-socks                    netcdf4                        pyphen                         tenacity
blinker                        huggingface-hub                networkx                       pyproject-hooks                tensorboard
blis                           identify                       nglview                        pyqt5                          tensorboard-data-server
boltons                        idna                           nltk                           pyqt5-qt5                      tensorflow
boltztrap2                     ifermi                         nodeenv                        pyqt5-sip                      tensorflow-io-gcs-filesystem
boto3                          ijson                          nose                           pyright                        termcolor
botocore                       imageio                        notebook                       pysocks                        terminado
bravado                        imagesize                      notebook-shim                  pytesseract                    textstat
bravado-core                   importlib-metadata             nougat-ocr                     pytest                         thinc
brotli                         importlib-resources            numba                          pytest-cov                     threadpoolctl
build                          iniconfig                      numpy                          pytest-mock                    tifffile
cachecontrol                   inquirer                       numpydoc                       pytest-split                   tiktoken
cachelib                       intel-cmplr-lib-ur             numpy-stl                      python-bidi                    timg
cachetools                     intel-openmp                   nvidia-cublas-cu11             python-dateutil                timm
castepxbin                     interfacemaster                nvidia-cublas-cu12             python-docx                    tinycss2
catalogue                      interpolation                  nvidia-cuda-cupti-cu11         python-dotenv                  tokenizers
cclib                          invoke                         nvidia-cuda-cupti-cu12         python-editor                  tokenize-rt
certifi                        ipykernel                      nvidia-cuda-nvrtc-cu11         python-fasthtml                toml
cffi                           ipython                        nvidia-cuda-nvrtc-cu12         python-json-logger             tomli
cfgv                           ipython-genutils               nvidia-cuda-runtime-cu11       python-levenshtein             tomlkit
cftime                         ipywidgets                     nvidia-cuda-runtime-cu12       python-markdown-math           toolz
charset-normalizer             isoduration                    nvidia-cudnn-cu11              python-multipart               torch
chart-studio                   isort                          nvidia-cudnn-cu12              python-socks                   torchdata
chemview                       itsdangerous                   nvidia-cufft-cu11              python-ulid                    torch-geometric
chgnet                         jarvis-tools                   nvidia-cufft-cu12              python-utils                   torchmetrics
cif2cell                       jedi                           nvidia-curand-cu11             python-xlib                    torchvision
clarabel                       jinja2                         nvidia-curand-cu12             pytorch-lightning              tornado
click                          jiter                          nvidia-cusolver-cu11           pytz                           tqdm
cloudpathlib                   jmespath                       nvidia-cusolver-cu12           pytz-deprecation-shim          trainstation
cmake                          jobflow                        nvidia-cusparse-cu11           pyupgrade                      traitlets
cmcrameri                      jobflow-remote                 nvidia-cusparse-cu12           pyusb                          traits
cnidcal                        joblib                         nvidia-ml-py3                  pywavelets                     transformers
colorama                       json2html                      nvidia-nccl-cu11               pyyaml                         trimesh
colorful                       json5                          nvidia-nccl-cu12               pyyaml-env-tag                 trio
colormath                      jsonlines                      nvidia-nvjitlink-cu12          pyzmq                          trio-websocket
comm                           jsonpointer                    nvidia-nvtx-cu11               qdldl                          triton
commonmark                     jsonref                        nvidia-nvtx-cu12               qtoolkit                       typer
confection                     jsonschema                     oauthlib                       qudida                         types-python-dateutil
contourpy                      jsonschema-specifications      openai                         rapidfuzz                      types-setuptools
coverage                       jupyter-cache                  openbabel-wheel                readchar                       typing
croniter                       jupyter-client                 opencv-python-headless         recommonmark                   typing-extensions
cryptography                   jupyter-core                   openmm                         referencing                    typing-inspection
crystal-toolkit                jupyter-events                 openmm-mdanalysis-reporter     regex                          tzdata
crystaltoolkit-extension       jupyterlab                     opt-einsum                     requests                       tzlocal
cssutils                       jupyterlab-code-formatter      optimade                       requests-futures               uc-micro-py
custodian                      jupyterlab-pygments            optree                         responses                      ujson
cvxpy                          jupyterlab-server              ordered-set                    retrying                       uncertainties
cycler                         jupyterlab-widgets             orjson                         rfc3339-validator              uncompyle6
cymem                          jupyter-lsp                    osqp                           rfc3986-validator              unidecode
cython                         jupyter-server                 outcome                        rfc3987                        universal-pathlib
dash                           jupyter-server-fileid          overrides                      rich                           uri-template
dash-core-components           jupyter-server-mathjax         packaging                      rjsmin                         urllib3
dash-html-components           jupyter-server-terminals       paginate                       rpds-py                        uv
dash-mp-components             jupyter-server-ydoc            palettable                     ruamel-yaml                    uvenv
dashscope                      jupyter-ydoc                   pandas                         ruamel-yaml-clib               uvicorn
dash-table                     jupytext                       pandocfilters                  ruff                           uvloop
datasets                       kaleido                        paramiko                       s3transfer                     validators
dateutils                      keras                          parso                          safehttpx                      vapory
debugpy                        kiwisolver                     patchelf                       safetensors                    virtualenv
decorator                      langcodes                      pathspec                       schedule                       vtk
deepdiff                       language-data                  pathy                          scikit-build                   wasabi
defusedxml                     lark                           pdfarranger                    scikit-image                   watchdog
deprecated                     latex2mathml                   pdfminer-six                   scikit-learn                   watchfiles
dgl                            latexcodec                     pebble                         scikit-opt                     wcwidth
dict2css                       lazy-loader                    periodictable                  scipdf-parser                  weasel
dill                           levenshtein                    pexpect                        scipy                          webcolors
diophantine                    libclang                       phonopy                        sconf                          webencodings
diskcache                      lightning                      pickleshare                    screeninfo                     websocket-client
distlib                        lightning-cloud                pillow                         scs                            websockets
distro                         lightning-utilities            pint                           seaborn                        werkzeug
dnspython                      linkify-it-py                  pip                            seekpath                       wheel
docker                         lit                            pip-tools                      selenium                       widgetsnbextension
docutils                       llvmlite                       platformdirs                   semantic-version               wrapt
domdf-python-tools             lobsterpy                      plotly                         send2trash                     wsproto
dscribe                        looseversion                   pluggy                         sentencepiece                  xdis
ecos                           lxml                           ply                            sentinels                      xkcd
edge-tts                       m3gnet                         pockets                        sentry-sdk                     xmltodict
email-validator                maggma                         pocketsphinx                   setuptools                     xxhash
emmet-core                     marimo                         pre-commit                     sgmllib3k                      yapf
entrypoints                    marisa-trie                    preshed                        shapely                        yarl
euporie                        markdown                       pretty-errors                  shellingham                    y-py
evdev                          markdown-it-py                 prometheus-client              shiboken2                      ypy-websocket
executing                      markupsafe                     prompt-toolkit                 shiboken6                      zhipuai
f90nml                         matgl                          propcache                      simplejson                     zipp
fabric                         matplotlib                     protobuf                       six                            
fastapi                        matplotlib-inline              psutil                         sixelcrop                      
fastcore                       matscipy                       ptyprocess                     smart-open               

@matthewhughes934
Copy link
Contributor

matthewhughes934 commented May 15, 2025

For other subcommands, say, install, show, etc., only the pip sub_command - works, pip sub_command won't give the expected options. I don't know if this is the expected behavior. See below for more details:

it looks like uninstall and show are special-cased to list installed packages if the current arg doesn't start with -

# special case: list locally installed dists for show and uninstall
and something similar for install
should_list_installables = (

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C: autocomplete Autocompletion in shells (pip completion) type: bug A confirmed bug or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants