Black formatter doesn't do anything

[dfarley1]

Environment data

• VS Code version: 1.38.1
• Extension version (available under the Extensions sidebar): 4.5
• OS and version: Ubuntu 18.04
• Python version (& distribution if applicable, e.g. Anaconda): 3.6
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: Black version 18.6b2

I'm using Remote-Containers to work inside a docker container.
Here's my /depot/.vscode/settings.json:

{
    "python.pythonPath": "/usr/bin/python3",
    // Where to pick up PYTHONPATH from
    "python.envFile": "${workspaceFolder}/.vscode/depot.env",
    // Lint python with flake8 (pyflakes + style)
    "python.linting.enabled": true,
    "python.linting.pylintEnabled": false,
    "python.linting.flake8Enabled": true,
    "python.linting.flake8Args": [
        //flake8 and black define the length differently
        "--max-line-length=81",
    ],
    // Format python with black line length 80
    "python.formatting.provider": "black",
    "python.formatting.blackArgs": [
        "--line-length",
        "80"
    ],
    "[python]": {
        "editor.formatOnSave": true,
        "editor.formatOnSaveTimeout": 5000,
    },
}

Expected behaviour

Saving a Python file or using "Format Document" command should format the document

Actual behaviour

Nothing happens to the file when saving or using the "Format Document" command.

If I manually run the command inside the container there's a fairly large diff output so black appears to be worked as expected and finding changes to make.
Running without --diff --quiet reformats the file and is reflected in VSCode.

Steps to reproduce:

1. Make a change to a Python file
2. Save the file

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

> /usr/bin/python3 -m black --line-length 80 --diff --quiet /depot/build/tools/install-clean.25efa79fc2195ad7261be01403cf4382
cwd: /depot
> /usr/bin/python3 -m black --line-length 80 --diff --quiet /depot/build/tools/install-clean.25efa79fc2195ad7261be01403cf4382
cwd: /depot
> /usr/bin/python3 -m flake8 --max-line-length=81 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s /depot/build/tools/install-clean
cwd: /depot
> /usr/bin/python3 -m flake8 --max-line-length=81 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s /depot/build/tools/install-clean
cwd: /depot
##########Linting Output - flake8##########
54,82,E,E501:line too long (93 > 81 characters)

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

console.ts:137 [Extension Host] Info Python Extension: 2019-09-27 12:15:36: Cached data exists getEnvironmentVariables, /depot/build/tools/install-clean
console.ts:137 [Extension Host] Info Python Extension: 2019-09-27 12:15:36: > /usr/bin/python3 -m black --line-length 80 --diff --quiet /depot/build/tools/install-clean.25efa79fc2195ad7261be01403cf4382
console.ts:137 [Extension Host] Info Python Extension: 2019-09-27 12:15:36: cwd: /depot
console.ts:137 [Extension Host] Info Python Extension: 2019-09-27 12:15:36: > /usr/bin/python3 -m black --line-length 80 --diff --quiet /depot/build/tools/install-clean.25efa79fc2195ad7261be01403cf4382
console.ts:137 [Extension Host] Info Python Extension: 2019-09-27 12:15:36: cwd: /depot
console.ts:137 [Extension Host] Info Python Extension: 2019-09-27 12:15:37: Cached data exists getEnvironmentVariables, /depot/build/tools/install-clean
console.ts:137 [Extension Host] Info Python Extension: 2019-09-27 12:15:37: > /usr/bin/python3 -m flake8 --max-line-length=81 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s /depot/build/tools/install-clean
console.ts:137 [Extension Host] Info Python Extension: 2019-09-27 12:15:37: cwd: /depot
console.ts:137 [Extension Host] Info Python Extension: 2019-09-27 12:15:37: > /usr/bin/python3 -m flake8 --max-line-length=81 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s /depot/build/tools/install-clean
console.ts:137 [Extension Host] Info Python Extension: 2019-09-27 12:15:37: cwd: /depot

[ericsnowcurrently]

Hi @dfarley1. Thanks for letting us know about this problem. The information you've provided is helpful.

The .25efa79fc2195ad7261be01403cf4382 suffix on the filename when black runs means we've created a temporary file because there are unsaved changed in the editor. We copy the contents of the editor into a temporary file and then run black against that. Do you have the same problem if you save the file and then run the "Format Document" command? (There won't be a temporary suffix.)

It's just a hunch, but this may be related to #6911.

[dfarley1]

I just tried it with a saved file and I'm getting the same results: "Format Document" ran, there was no suffix on the filename, but the file didn't change.
I do have a custom PYTHONPATH set via the python.envFile setting to include some internal libraries, could that effect anything?

[ericsnowcurrently]

@dfarley1, sorry for the delay. I'm still looking into it, but haven't been able to reproduce the problem so far. I'll keep investigating and let you know if I have any questions.

[frankyzhu]

I have the same issue, and I guess the problem lies in the parameter --diff. I don't know when it was added automatically.

[frankyzhu]

Everything works fine after upgrading vscode to 1.39.1.

[dfarley1]

I'm running 1.39.1 and it's still not working for me. Same behavior: python output shows black running but the file itself isn't changing at all. Tried with an already saved and an unsaved file.

[ericsnowcurrently]

@dfarley1, thanks for the update. I'm guessing that updating to the latest version of the extension hasn't helped either.

You mentioned before that you have a custom PYTHONPATH from a .env file. Do you have the problem if you remove the "python.envFile" setting?

Also, do you have any python-related settings in your user settings.json?

[dfarley1]

Yup, I'm on version 2019.10.44104. I tried removing the envFile option from my settings below and I get the same behavior.

...
    "python.pythonPath": "/usr/bin/python3",
    "python.envFile": "${workspaceFolder}/.vscode/depot.env",
    // Lint python with flake8 (pyflakes + style)
    "python.linting.enabled": true,
    "python.linting.pylintEnabled": false,
    "python.linting.flake8Enabled": true,
    "python.linting.flake8Args": [
        "--ignore=E203,E266,E402,W503",
        //flake8 and black define the length differently
        "--max-line-length=81",
        "--max-complexity=18",
        "--select=B,C,E,F,W,T4,B9"
    ],
    // Format python with black line length 80
    "python.formatting.provider": "black",
    "python.formatting.blackArgs": [
        "--line-length",
        "80",
    ],
    "[python]": {
        "editor.formatOnSave": true,
        "editor.formatOnSaveTimeout": 5000,
    },
...

[ericsnowcurrently]

Let's try simplifying. What happens if you using the following workspace settings.json?

{
    "python.pythonPath": "/usr/bin/python3",
    "python.linting.enabled": false,
    "python.formatting.provider": "black",
    "[python]": {
        "editor.formatOnSave": true,
        "editor.formatOnSaveTimeout": 5000,
    }
}

Also, what is the command that the extension runs? (check the "Python" output panel)

[dfarley1]

Here's the output I got when hitting Save on a badly formatted file, the file did not get fixed

> /usr/bin/python3 -m black --diff --quiet .../util.py.d32f5e35449801764ca300a6b407dc1f.py
cwd: /depot
> /usr/bin/python3 -m black --diff --quiet .../util.py.d32f5e35449801764ca300a6b407dc1f.py
cwd: /depot

Here's the output when I hit Format on an already saved file, the file did not get fixed

> /usr/bin/python3 -m black --diff --quiet .../util.py
cwd: /depot
> /usr/bin/python3 -m black --diff --quiet .../util.py
cwd: /depot

Here's the output if I run the command in the VSCode terminal (inside the container), the file did not get fixed (due to --diff) but I get some output

danf@docker ~ $ /usr/bin/python3 -m black --diff --quiet .../util.py
--- .../util.py	2019-11-13 21:20:10.945857 +0000
+++ .../util.py	2019-11-13 21:21:34.815809 +0000

<diffy stuff here>

danf@docker ~ $ 

[ericsnowcurrently]

@dfarley1, I may have been able to reproduce the problem. Here's what I did:

1. created a new folder
2. ran "Remote-Containers: Open Folder in Container" command
3. created the workspace settings.json from above
4. created spam.py
5. saved with the following text:

import sys, \
    os

...

At this point black worked fine and the text was reformatted. So then I undid the reformatting, added some invalid syntax, and saved. The code was not reformatted this time. Nor was there any error. So it appears that the extension is hiding black failures (which were actually caught by the linter). When I run the command manually at the terminal I get a failure message about the syntax error.

So it sounds like the failure I ran into isn't the same as yours. From what you have indicated, you do not have any bad syntax and you are actually getting a diff when running the command manually. Is that correct?

Here are a couple more questions:

• do you have the same problem when working locally (not in a container)?
• do you have some example code that I could use to reproduce the problem?

[dfarley1]

I followed your steps using a new folder and I'm getting the same behavior. Black runs but does not fix the file.

danf@docker ~ $ cat test/spam.py 
import sys, \
    os

class asdf(int):
    pass

From what you have indicated, you do not have any bad syntax and you are actually getting a diff when running the command manually. Is that correct?

Correct, linting works and a real diff appears.

danf@docker ~ $ /usr/bin/python3 -m black --diff --quiet ~/test/spam.py
--- /home/danf/test/spam.py	2019-11-21 00:03:25.897859 +0000
+++ /home/danf/test/spam.py	2019-11-21 01:42:12.198430 +0000
@@ -1,6 +1,6 @@
-import sys, \
-    os
+import sys, os
+
 
 class asdf(int):
     pass

do you have the same problem when working locally (not in a container)?

No, I did the same steps without a container and formatting via VSCode works as expected.

[ericsnowcurrently]

So following the steps I enumerated above, you were able to reproduce the problem? If so then there is something funny going on. What version of docker are you using?

[dfarley1]

Docker version 19.03.5, build 633a0ea838

I've updated VSCode and extensions to the latest versions, respawned my container, and rebooted my host PC a number of times while testing so there's something systemic going on.

[ericsnowcurrently]

Until we can reproduce this there isn't a lot we can do, unfortunately. Do you see anything out of the ordinary in any of the output panels (not just the Python-related ones)? Which output panels can you see?

[dfarley1]

I can see Tasks, Extensions, Python, Git, Python Test Log, Gitlens, and logs for Extension Host, Remote Server, Window, Main, Shared, and Remote Extension Host.

All of the panels look OK to me, the only error even during a Reload Window command is from Log (Window) and it doesn't seem related:

[2019-11-25 14:04:48.585] [renderer1] [error] TreeError [MarkersPanel] Tree element not found: [object Object]: Error: TreeError [MarkersPanel] Tree element not found: [object Object]
    at t.ObjectTreeModel.getElementLocation (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:133:120)
    at t.ObjectTreeModel.isCollapsed (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:132:250)
    at W.isCollapsed (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1043:391)
    at z.revealMarkersForCurrentActiveEditor (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:5926:628)
    at z.autoReveal (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:5926:485)
    at z.onActiveEditorChanged (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:5923:776)
    at file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:5922:754
    at file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:111:709
    at c.fire (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:115:286)
    at e.doEmitActiveEditorChangeEvent (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:4909:172)
    at e.handleActiveEditorChange (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:4908:992)
    at file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:4909:282
    at c.fire (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:115:286)
    at file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:5569:875
    at async e.openEditor (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:5568:750)

[ericsnowcurrently]

Yeah, that doesn't look related.

[ericsnowcurrently]

Is this a container that you created via VS Code (command Remote-Containers: Open Folder in Container)? What kind of container configuration did you pick? I ask because you originally reported you are using Python 3.6, but I'm not sure that any of the pre-packaged configurations have that version installed.

For my example code I picked the "Python 3" (python:3) configuration. That created a container with Python 3.8 ready to go. I installed black (18.6v2) there and triggered formatting. It correctly re-formatted the file. I also tried it with the latest release of black, which also worked.

[ericsnowcurrently]

Also, I made sure add "python.jediEnabled": true to my settings.json, since you are using Jedi (by default).

[ericsnowcurrently]

And just to be clear, when you ran my example code, you did it in a new folder (ergo a new container), right? I've noticed that it's tricky to get a fresh container when using VS Code to create it.

[dfarley1]

Is this a container that you created via VS Code (command Remote-Containers: Open Folder in Container)? What kind of container configuration did you pick?

I did not create the container using VSCode, I've been trying to "integrate" VSCode with a pre-existing build system that lives inside a container based on Gentoo. I figured if VSCode can edit files, run flake8, and do everything else then it should be fine to run black as well.

Also, I made sure add "python.jediEnabled": true to my settings.json, since you are using Jedi (by default).

I've never touched anything related to Jedi, why do you say I'm using it and why would it affect anything?

And just to be clear, when you ran my example code, you did it in a new folder (ergo a new container), right? I've noticed that it's tricky to get a fresh container when using VS Code to create it.

No, I did not create a new container. I only created a new workspace inside the container that our build spawns.

[ericsnowcurrently]

Regarding Jedi, the "python.jediEnabled" setting controls whether the extension uses Jedi or the language server. By default it is set to "true". Just to be sure, I set it explicitly.

I'm not sure it would have any effect, but I'm trying to get everything exactly the same. :)

[ericsnowcurrently]

Do you have the same problem in a container that VS Code creates?

[dfarley1]

I haven't had the chance to try out my own container. I'm not sure what caused it but at some point over the last week formatting started working. I know VSCode itself had an update, did the extension? I don't think anything in our container environment changed.

[ericsnowcurrently]

The last extension release was in November. So is everything working for you now? If so, feel free to close this issue.