Getting "permission denied" when trying to commit after setting up precommit hooks

[dhersz]

Hi!

I followed the instructions on how to install and use the package as listed in the readme, but I have not been able to commit after setting up .pre-commit-config.yaml.

I just want to use two specific hooks, readme-rmd-rendered and codemeta-description-updated, so this is how my .pre-commit-config.yaml looks like:

# All available hooks: https://pre-commit.com/hooks.html
# R specific hooks: https://github.com/lorenzwalthert/precommit
repos:
-   repo: https://github.com/lorenzwalthert/precommit
    rev: v0.1.3
    hooks:
    -   id: readme-rmd-rendered
    -   id: codemeta-description-updated
 

Then I tried to commit these changes (just precommit-related) and I'm getting this error message:

$ git commit -m "creates pre-commit hooks"
/usr/bin/env: ‘python’: Permission denied

The hooks actions should not be a problem here, because both README.md and codemeta.json are more recent than README.Rmd and DESCRIPTION.

I usually commit through Git Bash, so I figured out it might be something with it. Then I tried committing through RStudio, but I stumbled upon the same error.

I finally had to disable precommit using precommit::uninstall_precommit("repo"), which allowed me to commit back again. I haven't uninstalled it user-wise, but I tried committing in a non-R related repo and it seemed to work fine.

Not sure if I have messed up when setting the pre-commit-hooks yaml up, but I could not go past this error.

---

Some session info:

> devtools::session_info()
- Session info ---------------------------------------------------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 4.0.3 (2020-10-10)
 os       Windows 10 x64              
 system   x86_64, mingw32             
 ui       RStudio                     
 language (EN)                        
 collate  Portuguese_Brazil.1252      
 ctype    Portuguese_Brazil.1252      
 tz       America/Sao_Paulo           
 date     2021-01-29                  

- Packages -------------------------------------------------------------------------------------------------------------------------------------------
 ! package     * version    date       lib source        
   askpass       1.1        2019-01-13 [1] CRAN (R 4.0.2)
   assertthat    0.2.1      2019-03-21 [1] CRAN (R 4.0.2)
   backports     1.2.0      2020-11-02 [1] CRAN (R 4.0.3)
   cachem        1.0.1      2021-01-21 [1] CRAN (R 4.0.3)
   callr         3.5.1      2020-10-13 [1] CRAN (R 4.0.3)
   checkmate     2.0.0      2020-02-06 [1] CRAN (R 4.0.2)
   class         7.3-18     2021-01-24 [1] CRAN (R 4.0.3)
   classInt      0.4-3      2020-04-07 [1] CRAN (R 4.0.2)
   cli           2.2.0      2020-11-20 [1] CRAN (R 4.0.3)
   crayon        1.3.4      2017-09-16 [1] CRAN (R 4.0.2)
   credentials   1.3.0      2020-07-21 [1] CRAN (R 4.0.3)
   data.table    1.13.6     2020-12-30 [1] CRAN (R 4.0.3)
   DBI           1.1.1      2021-01-15 [1] CRAN (R 4.0.3)
   desc          1.2.0      2018-05-01 [1] CRAN (R 4.0.2)
   devtools      2.3.2      2020-09-18 [1] CRAN (R 4.0.3)
   dplyr         1.0.3      2021-01-15 [1] CRAN (R 4.0.3)
   e1071         1.7-4      2020-10-14 [1] CRAN (R 4.0.3)
   ellipsis      0.3.1      2020-05-15 [1] CRAN (R 4.0.2)
   fansi         0.4.2      2021-01-15 [1] CRAN (R 4.0.3)
   fastmap       1.1.0      2021-01-25 [1] CRAN (R 4.0.3)
   fs            1.5.0      2020-07-31 [1] CRAN (R 4.0.2)
   generics      0.1.0      2020-10-31 [1] CRAN (R 4.0.3)
   gert          1.1.0      2021-01-25 [1] CRAN (R 4.0.3)
   git2r         0.28.0     2021-01-10 [1] CRAN (R 4.0.3)
   glue          1.4.2      2020-08-27 [1] CRAN (R 4.0.3)
 P gtfstools   * 0.0.0.9000 2021-01-28 [?] local         
   here          1.0.1      2020-12-13 [1] CRAN (R 4.0.3)
   jsonlite      1.7.2      2020-12-09 [1] CRAN (R 4.0.3)
   KernSmooth    2.23-18    2020-10-29 [1] CRAN (R 4.0.3)
   lattice       0.20-41    2020-04-02 [2] CRAN (R 4.0.3)
   lifecycle     0.2.0      2020-03-06 [1] CRAN (R 4.0.2)
   lwgeom        0.2-5      2020-06-12 [1] CRAN (R 4.0.2)
   magrittr      2.0.1      2020-11-17 [1] CRAN (R 4.0.3)
   Matrix        1.2-18     2019-11-27 [1] CRAN (R 4.0.3)
   memoise       2.0.0      2021-01-26 [1] CRAN (R 4.0.3)
   openssl       1.4.3      2020-09-18 [1] CRAN (R 4.0.3)
   pillar        1.4.7      2020-11-20 [1] CRAN (R 4.0.3)
   pkgbuild      1.2.0      2020-12-15 [1] CRAN (R 4.0.3)
   pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.0.2)
   pkgload       1.1.0      2020-05-29 [1] CRAN (R 4.0.2)
   precommit     0.1.3      2020-10-10 [1] CRAN (R 4.0.3)
   prettyunits   1.1.1      2020-01-24 [1] CRAN (R 4.0.2)
   processx      3.4.5      2020-11-30 [1] CRAN (R 4.0.3)
   ps            1.5.0      2020-12-05 [1] CRAN (R 4.0.3)
   purrr         0.3.4      2020-04-17 [1] CRAN (R 4.0.2)
   R6            2.5.0      2020-10-28 [1] CRAN (R 4.0.2)
   rappdirs      0.3.2      2021-01-27 [1] CRAN (R 4.0.3)
   Rcpp          1.0.6      2021-01-15 [1] CRAN (R 4.0.3)
   remotes       2.2.0      2020-07-21 [1] CRAN (R 4.0.3)
   reticulate    1.18       2020-10-25 [1] CRAN (R 4.0.3)
   rlang         0.4.10     2020-12-30 [1] CRAN (R 4.0.3)
   rprojroot     2.0.2      2020-11-15 [1] CRAN (R 4.0.3)
   rstudioapi    0.13       2020-11-12 [1] CRAN (R 4.0.3)
   sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 4.0.2)
   sf            0.9-7      2021-01-06 [1] CRAN (R 4.0.3)
   sfheaders     0.4.0      2020-12-01 [1] CRAN (R 4.0.3)
   sys           3.4        2020-07-23 [1] CRAN (R 4.0.2)
   testthat    * 3.0.1      2020-12-17 [1] CRAN (R 4.0.3)
   tibble        3.0.5      2021-01-15 [1] CRAN (R 4.0.3)
   tidyselect    1.1.0      2020-05-11 [1] CRAN (R 4.0.2)
   units         0.6-7      2020-06-13 [1] CRAN (R 4.0.2)
   usethis       2.0.0      2020-12-10 [1] CRAN (R 4.0.3)
   vctrs         0.3.6      2020-12-17 [1] CRAN (R 4.0.3)
   withr         2.4.1      2021-01-26 [1] CRAN (R 4.0.3)
   yaml          2.2.1      2020-02-01 [1] CRAN (R 4.0.0)
   zip           2.1.1      2020-08-27 [1] CRAN (R 4.0.3)

[1] C:/Users/Usuario/Documents/R/win-library/4.0
[2] C:/Program Files/R/R-4.0.3/library

[lorenzwalthert]

Thanks for your report. What was the installation method you chose and from where did you execute it? Note that for pip install pre-commit, we only tested with the --user flag. From your permission denied error, it seems pre-commit tries to use a non-user python installation. If you have admin privileges, you can also try to make /usr/bin/env/python executable by the user.

A potential (other) problem is that this is a python 2.x version, and pre-commit does not support it anymore.

Edit: The problem should be resolved by upgrading the hooks, which are now of language: r (for proper dependency management in a renv that {precommit} manages) instead of language: script (which depends on the shebang). Run precommit::autoupdate().

[dhersz]

I used the miniconda method:

reticulate::install_miniconda()
precommit::install_precommit()

I tried installing the most recent python 3.x version and reinstalling miniconda and precommit and now I seem to get a different error log:

$ git commit -m "adds pre-commit hooks"
Traceback (most recent call last):
  File "C:\Users\Usuario\AppData\Local\r-miniconda\envs\r-precommit\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Usuario\AppData\Local\r-miniconda\envs\r-precommit\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Usuario\AppData\Local\r-miniconda\envs\r-precommit\lib\site-packages\pre_commit\__main__.py", line 1, in <module>
    from pre_commit.main import main
  File "C:\Users\Usuario\AppData\Local\r-miniconda\envs\r-precommit\lib\site-packages\pre_commit\main.py", line 13, in <module>
    from pre_commit.commands.autoupdate import autoupdate
  File "C:\Users\Usuario\AppData\Local\r-miniconda\envs\r-precommit\lib\site-packages\pre_commit\commands\autoupdate.py", line 20, in <module>
    from pre_commit.store import Store
  File "C:\Users\Usuario\AppData\Local\r-miniconda\envs\r-precommit\lib\site-packages\pre_commit\store.py", line 4, in <module>
    import sqlite3
  File "C:\Users\Usuario\AppData\Local\r-miniconda\envs\r-precommit\lib\sqlite3\__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "C:\Users\Usuario\AppData\Local\r-miniconda\envs\r-precommit\lib\sqlite3\dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: DLL load failed while importing _sqlite3: N▒o foi poss▒vel encontrar o m▒dulo especificado.

(the last line means "couldn't find the specified module")

[lorenzwalthert]

I used the miniconda method:

ok. That is surprising.

I tried installing the most recent python 3.x version

I don't suggest doing that. The above installation commands should handle the python version.

Your new error looks like pre-commit/pre-commit#1329. Also, are you using the most recent {reticulate} version to install miniconda? Because you might install an old conda version otherwise with reticulate::install_miniconda(). Alternatively, I'd try pip install pre-commit --user.

[dhersz]

Great, it worked with pip3 install pre-commit --user! Thank you very much for the attention and patience @lorenzwalthert.

(P.S. Just to make it clear, I was in fact using the most recent reticulate version)

[dhersz]

Perhaps I have closed too early...

I tried testing the hooks and I have received a couple executable not found messages:

$ git commit -m "bumps version (to test pre-commit hooks)"
git push codemeta-description-updated.............................................Failed
- hook id: codemeta-description-updated
- exit code: 1

Executable `/bin/bash` not found

use-tidy-description.....................................................Failed
- hook id: use-tidy-description
- exit code: 1

Executable `Rscript` not found

readme-rmd-rendered..................................(no files to check)Skipped
Don't commit common R artifacts......................(no files to check)Skipped

Ever seen this before?

[lorenzwalthert]

Did you try in a fresh terminal session? Also, is it git bash or in RStudio?

[dhersz]

I initially tried with Git Bash and got those errors.

Trying with RStudio makes the Rscript error go away, but the bash problem still persists and I get a warning due to the package encoding (not sure, but not pre-commit related lol):

>>> C:/Program Files/Git/bin/git.exe commit -F C:/Users/Usuario/AppData/Local/Temp/RtmpOUdQUY/git-commit-message-27e8377d1944.txt
[WARNING] Unstaged files detected.
[INFO] Stashing unstaged files to C:\Users\Usuario\.cache\pre-commit\patch1612032021.
codemeta-description-updated.............................................Failed
- hook id: codemeta-description-updated
- exit code: 1

Executable `/bin/bash` not found
use-tidy-description.....................................................Failed
- hook id: use-tidy-description
- files were modified by this hook

During startup - Warning message:
Setting LC_CTYPE=en_US.UTF-8 failed 
<U+221A> Setting active project to 'C:/Users/Usuario/Documents/repos/gtfstools'

readme-rmd-rendered..................................(no files to check)Skipped
Don't commit common R artifacts......................(no files to check)Skipped

Trying again with Git Bash, after adding R/bin to the system PATH variable:

$ git commit -m "bumps version (to test pre-commit hooks)"
codemeta-description-updated.............................................Failed
- hook id: codemeta-description-updated
- exit code: 1

Executable `/bin/bash` not found

use-tidy-description.....................................................Failed
- hook id: use-tidy-description
- exit code: 1

Error in loadNamespace(name) : there is no package called 'usethis'
Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted

readme-rmd-rendered..................................(no files to check)Skipped
Don't commit common R artifacts......................(no files to check)Skipped

I must say that bash is recognized as a valid command when I run it in the Windows terminal.

[lorenzwalthert]

install.packages('usethis') and your second error should go.

[lorenzwalthert]

I must say that bash is recognized as a valid command when I run it in the Windows terminal.

Ok, but it is expected to by at /bin/bash. Either deactivate the codemeta hook if you don't need it, otherwise, you might be able to symlink bash at /bin/bash.

[lorenzwalthert]

Try which bash to get the path to your bash executable.

[dhersz]

install.packages('usethis') and your second error should go.

I have it installed already. I managed to solve the problem by adding a environment variable called R_LIBS_USER with the path to where my packages are installed. RStudio automatically detected this package, but when running Rscript inside Git Bash it wasn't detecting. It works fine now.

(btw, I noticed that use-tidy-description doesn't auto stage the changes it makes to DESCRIPTION. is that intended?)

Try which bash to get the path to your bash executable.

$ which bash
/usr/bin/bash

[lorenzwalthert]

adding a environment variable called R_LIBS_USER

Sounds like a hack but if it works ... xD

(btw, I noticed that use-tidy-description doesn't auto stage the changes it makes to DESCRIPTION. is that intended?)

Yes. All hooks of the pre-commit framework work like this. That way, you can inspect the diff first.

For the bash executable, you can for example create an alias or similar to make it work. Do you really want this hook? If not, just deactivate it. Sorry I can't provide more guidance but you seem to be the first person to have this issue.

[dhersz]

No worries, I'll make a local copy with the updated shebang.

I imagine that fixing something that is broken for a single person may very well be a hassle. If someone else faces this same problem in the future, though, I think this Stack Exchange thread might help (I don't really have any experience with shell scripting, but it seems that the topic discussed address this problem).

Cheers, and thank you again!

[lorenzwalthert]

Not sure you can make a local copy to fix the shebang. Because pre-commit will still use this GitHub repo to run the hooks. I saw this post before, maybe we should use this approach. We use it with other books already it seems. Because if all people in windows have this problem, it might be worth fixing.

[lorenzwalthert]

Also, I am surprised because even usethis' pre-commit hook for the README uses this approach with /bin/bash and apparently no one had issues so far (when checking old issues):

https://github.com/r-lib/usethis/blob/master/inst/templates/readme-rmd-pre-commit.sh

[dhersz]

Hmm I see. I initially thought about doing something similar as you do in the .pre-commit-config.yaml of this very repo, using the repo: local tag and pointing to a local path in the entry argument (and then basically copying the specific parts of the hooks I want from .pre-commit-hooks.yaml). But yeah, I have no experience with pre-commit hooks, do you think that would work?

Also, I am surprised because even usethis' pre-commit hook for the README uses this approach with /bin/bash and apparently no one had issues so far:

https://github.com/r-lib/usethis/blob/master/inst/templates/readme-rmd-pre-commit.sh

Yeah, I'm not sure to what extent this error I'm getting is Windows-related or just myself-related lol

[lorenzwalthert]

Ok. Yes, you can do a local hook. Just make sure your script has the permission to be executed (try chmod u+rwx /path/to/scrip or similar) or do it in the windows property GUI (after right-clicking on the file). You can find a few examples of how local hooks work in this repo. Other option is just to place the bash executable in the /bin directory or try to make an alias with Windows GUI. Might also work and has better portability.

[dhersz]

Just a quick update, I ended up creating a repo with the hooks with updated shebang and I refer to this repo in the config.yaml. It works like a charm :)

I figured out it would be better than to include them in a inst/bin directory in every package I'm writing.

Here is the link to the repo, I made sure to give credits to you. Here's a reproducible code to use the hooks, in case you ever decide to test it or anyone stumbles across the same problems I did (mod stands for modified):

repos:
-   repo: https://github.com/dhersz/dhmisc
    rev: v0.0.0.9005
    hooks:
    -   id: mod-codemeta-description-updated
    -   id: mod-readme-rmd-rendered

Again, thank you very much! Cheers

[lorenzwalthert]

We'll rewrite this in R at some point since it will be required to make hooks work on CI/CD as outlined in #282.

[lorenzwalthert]

@dhersz these path problems should be resolved with the latest version of {precommit}, because R is now a supported language in pre-commit. Can you try it? You need to update all components as described in the README.md. Well wait until #350 is merged and released, then it should also work for the code meta hook.

[lorenzwalthert]

Done.