Stub method being overshadowed by module

[malmaud]

With the new PyTorch type stubs (pytorch/pytorch#12500), the stub for the tensor method seems to not be utilized by intellisense (example below). I think this is related to the existence of a module called tensor.py in the PyTorch package - if I delete that file, then the stub is utilized. Maybe the language server is somehow getting confused between whether torch.tensor should be inferred to refer to the module or the method defined in the stub. In this case, it should refer to the stub.

The stub for tensor indicates it has a Tensor return type, but we see vscode not showing Tensor methods here:

Meanwhile the similar as_tensor method, which has the same return type, is working:

Here are the stub definitions:

def tensor(data: Any, dtype: Optional[_dtype]=None, device: Union[_device, str, None]=None, requires_grad: bool=False) -> Tensor: ...

def as_tensor(data: Any, dtype: _dtype=None, device: Optional[_device]=None) -> Tensor: ...

I don't have a minimal example at the moment. PyTorch is a big package and in my simple attempts to reduce the problem to a minimal version, VSCode did the right thing, so it's hard for me to tell what exactly is going wrong.

This is with Python 3.7, extension version 2019.5.5433 and VSCode 1.31.1.

[jakebailey]

I'm checking this against the LS in master (very different than the released one) which does a much better job with stubs, and can at least do:

However, neither torch.tensor nor torch.as_tensor are things which appear in the torch module's __all__ list, which means that they aren't being exposed:

__all__ = [
    'typename', 'is_tensor', 'is_storage', 'set_default_tensor_type',
    'set_rng_state', 'get_rng_state', 'manual_seed', 'initial_seed',
    'save', 'load', 'set_printoptions', 'chunk', 'split', 'stack', 'matmul',
    'no_grad', 'enable_grad', 'rand', 'randn',
    'DoubleStorage', 'FloatStorage', 'LongStorage', 'IntStorage',
    'ShortStorage', 'CharStorage', 'ByteStorage',
    'DoubleTensor', 'FloatTensor', 'LongTensor', 'IntTensor',
    'ShortTensor', 'CharTensor', 'ByteTensor', 'Tensor',
]

https://github.com/pytorch/pytorch/blob/master/torch/__init__.py#L19

Are you on a different version of Torch which exposes those functions? I'm testing with:

• Python 3.7, venv
• pip installed Torch 1.0 non-CUDA version from: https://pytorch.org/get-started/locally/

[jakebailey]

Oh. Gross.

from torch._C import *

__all__ += [name for name in dir(_C)
            if name[0] != '_' and
            not name.endswith('Base')]

[malmaud]

Thanks for looking into this!

Ya, it's because of dynamic code like that that a type stub was added to aid code completion. You'll see a __init__.pyi in the 'torch' package folder, and that stub explicitly declares a def tensor(...).

I opened this issue because I thought the language server would use the information in the stub, and it does seem to with the exception of the tensor method. It's on the nightly pytorch that I see as_tensor is being suggested as a completion while tensor is not, despite them both being in the stub file.

I installed the nightly with pip install torch_nightly -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html.

[jakebailey]

The version of torch I have installed doesn't have those two functions in the stub, so I'll have to switch up and see. With some __all__ filtering removed, I can see loads of Torch functions already.

Just to say so, the visual difference between -> Tensor and -> 'Tensor' is visual only; the tooltips in the new language server are essentially copies from the declaration to the output, but the analysis itself parses out the string and does something with it.

[jakebailey]

Alright, testing with the nightly build of torch:

I'm going to close this issue, as it's working as expected in the new LS (which should be released into daily and beta soon).

[malmaud]

Perfect, thanks.

…

On Wed, Mar 6, 2019 at 1:41 PM Jake Bailey ***@***.***> wrote: Closed #690 <#690>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#690 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA8Svalw-rMq8ib89vt27h5GV-nVYyFqks5vUDYQgaJpZM4bapYE> .

[jakebailey]

0.2.16 is now available on beta channel. You can set this in your VS Code settings.json to get it:

"python.analysis.downloadChannel": "beta"

[malmaud]

Confirmed that switching to the beta channel fixes things on my end too.