Skip to content

Add stubs for tree_sitter #8533

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Aug 17, 2022
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pyrightconfig.stricter.json
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@
"stubs/SQLAlchemy",
"stubs/stripe",
"stubs/tqdm",
"stubs/tree-sitter",
"stubs/tzlocal/tzlocal/utils.pyi",
"stubs/ttkthemes",
"stubs/urllib3",
Expand Down
31 changes: 31 additions & 0 deletions stubs/tree-sitter/@tests/stubtest_allowlist.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# "self" argument is missing when stubtest inspects these methods
tree_sitter.Node.child_by_field_id
tree_sitter.Node.child_by_field_name
tree_sitter.Node.sexp
tree_sitter.Node.walk
tree_sitter.Parser.parse
tree_sitter.Parser.set_language
tree_sitter.Tree.edit
tree_sitter.Tree.walk
tree_sitter.TreeCursor.current_field_name
tree_sitter.TreeCursor.goto_first_child
tree_sitter.TreeCursor.goto_next_sibling
tree_sitter.TreeCursor.goto_parent
tree_sitter.binding.Node.child_by_field_id
tree_sitter.binding.Node.child_by_field_name
tree_sitter.binding.Node.sexp
tree_sitter.binding.Node.walk
tree_sitter.binding.Parser.parse
tree_sitter.binding.Parser.set_language
tree_sitter.binding.Query.captures
tree_sitter.binding.Query.matches
tree_sitter.binding.Tree.edit
tree_sitter.binding.Tree.walk
tree_sitter.binding.TreeCursor.current_field_name
tree_sitter.binding.TreeCursor.goto_first_child
tree_sitter.binding.TreeCursor.goto_next_sibling
tree_sitter.binding.TreeCursor.goto_parent

# Runtime takes *args and **kwargs and ignores them. Passing arguments is most likely a mistake.
tree_sitter.Parser.__init__
tree_sitter.binding.Parser.__init__
4 changes: 4 additions & 0 deletions stubs/tree-sitter/METADATA.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
version = "0.20.*"

[tool.stubtest]
ignore_missing_stub = false
16 changes: 16 additions & 0 deletions stubs/tree-sitter/tree_sitter/__init__.pyi
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
from _typeshed import Incomplete, StrPath
from collections.abc import Sequence

# Query is missing at runtime for some reason
from tree_sitter.binding import Node as Node, Parser as Parser, Tree as Tree, TreeCursor as TreeCursor

class Language:
@staticmethod
def build_library(output_path: str, repo_paths: Sequence[StrPath]): ...
name: str
lib: Incomplete
language_id: int
# library_path is passed into ctypes LoadLibrary
def __init__(self, library_path: str, name: str) -> None: ...
def field_id_for_name(self, name): ...
def query(self, source): ...
95 changes: 95 additions & 0 deletions stubs/tree-sitter/tree_sitter/binding.pyi
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
from _typeshed import Incomplete
from typing import ClassVar
from typing_extensions import final

from tree_sitter import Language

@final
class Node:
@property
Copy link
Member

@JelleZijlstra JelleZijlstra Aug 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a bunch of other properties: https://github.com/tree-sitter/py-tree-sitter/blob/7dc704fa37057c685fe2d86611f318146c95c34d/tree_sitter/binding.c#L501. I didn't check them all but you're missing at least id and named_children. But we don't necessarily need to include those if they're undocumented.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These don't exist in the latest released version of py-tree-sitter (0.20.0).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'll teach me to look at the master branch :)

def start_byte(self) -> int: ...
@property
def start_point(self) -> tuple[int, int]: ...
@property
def end_byte(self) -> int: ...
@property
def end_point(self) -> tuple[int, int]: ...
@property
def has_changes(self) -> bool: ...
@property
def has_error(self) -> bool: ...
@property
def is_missing(self) -> bool: ...
@property
def is_named(self) -> bool: ...
@property
def child_count(self) -> int: ...
@property
def named_child_count(self) -> bool: ...
@property
def children(self) -> list[Node]: ...
@property
def next_named_sibling(self) -> Node | None: ...
@property
def next_sibling(self) -> Node | None: ...
@property
def parent(self) -> Node | None: ...
@property
def prev_named_sibling(self) -> Node | None: ...
@property
def prev_sibling(self) -> Node | None: ...
@property
def text(self) -> bytes: ...
@property
def type(self) -> str: ...
__hash__: ClassVar[None] # type: ignore[assignment]
def child_by_field_id(self, __id: int) -> Node | None: ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are more methods: children_by_field_id, children_by_field_name, field_name_for_child

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are not in the latest released version.

def child_by_field_name(self, __name: str) -> Node | None: ...
def sexp(self) -> str: ...
def walk(self) -> TreeCursor: ...
def __eq__(self, other: object) -> bool: ...
def __ne__(self, other: object) -> bool: ...
# There are __ge__, __gt__, __le__, __lt__ methods but they seem to always return False
#
# >>> n
# <Node kind=call, start_point=(0, 0), end_point=(0, 14)>
# >>> n >= "", n <= "", n >= 0, n <= 0, n >= (0,0), n <= (0,0)
# (False, False, False, False, False, False)

@final
class Parser:
# At runtime, Parser(1, 2, 3) ignores the arguments, but that's most likely buggy code
def __init__(self) -> None: ...
def parse(self, source: bytes, old_tree: Tree | None = ..., keep_text: bool = ...) -> Tree: ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version in pypi is more strict:

>>> parser.parse(source=print)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: First argument to parse must be bytes
>>> parser.parse(source=bytearray(b'asd'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: First argument to parse must be bytes

def set_language(self, __language: Language) -> None: ...

@final
class Query:
def captures(self, *args, **kwargs): ...
def matches(self, *args, **kwargs): ...

@final
class Tree:
@property
def root_node(self) -> Node: ...
@property
def text(self) -> bytes: ...
def edit(
self,
start_byte: int,
old_end_byte: int,
new_end_byte: int,
start_point: tuple[int, int],
old_end_point: tuple[int, int],
new_end_point: tuple[int, int],
) -> None: ...
def walk(self) -> TreeCursor: ...

@final
class TreeCursor:
@property
def node(self) -> Node: ...
def current_field_name(self) -> None | Incomplete: ...
def goto_first_child(self) -> bool: ...
def goto_next_sibling(self) -> bool: ...
def goto_parent(self) -> bool: ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def goto_parent(self) -> bool: ...
def goto_parent(self) -> bool: ...
def copy(self) -> TreeCursor: ...

One more

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't exist in the released version.