Skip to content

Add stubs for tree_sitter #8533

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Aug 17, 2022
Merged

Add stubs for tree_sitter #8533

merged 24 commits into from
Aug 17, 2022

Conversation

Akuli
Copy link
Collaborator

@Akuli Akuli commented Aug 11, 2022

tree-sitter is a library for parsing various languages. I just merged a pull request that uses it into https://github.com/Akuli/porcupine/, so we should have a basic level of mypy_primer coverage.

@github-actions

This comment has been minimized.

1 similar comment
@github-actions

This comment has been minimized.

@Akuli Akuli marked this pull request as draft August 11, 2022 19:29
@github-actions

This comment has been minimized.

1 similar comment
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

1 similar comment
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@Akuli Akuli marked this pull request as ready for review August 11, 2022 20:19
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly useless on this one as I'm not much of a C programmer, but here's a review for the bit where I'm confident I can read the source code correctly :)

@Akuli
Copy link
Collaborator Author

Akuli commented Aug 12, 2022

I didn't read the source code when making this PR, I just experimented on the interactive prompt. But installing py-tree-sitter locally requires you to have a C compiler, so getting it to work at all on Windows isn't easy either :)

@github-actions

This comment has been minimized.

2 similar comments
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Member

@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a look at the C code and have a few pieces of feedback


@final
class Node:
@property
Copy link
Member

@JelleZijlstra JelleZijlstra Aug 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a bunch of other properties: https://github.com/tree-sitter/py-tree-sitter/blob/7dc704fa37057c685fe2d86611f318146c95c34d/tree_sitter/binding.c#L501. I didn't check them all but you're missing at least id and named_children. But we don't necessarily need to include those if they're undocumented.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These don't exist in the latest released version of py-tree-sitter (0.20.0).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'll teach me to look at the master branch :)

@property
def type(self) -> str: ...
__hash__: ClassVar[None] # type: ignore[assignment]
def child_by_field_id(self, __id: int) -> Node | None: ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are more methods: children_by_field_id, children_by_field_name, field_name_for_child

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are not in the latest released version.

class Parser:
# At runtime, Parser(1, 2, 3) ignores the arguments, but that's most likely buggy code
def __init__(self) -> None: ...
def parse(self, source: bytes, old_tree: Tree | None = ..., keep_text: bool = ...) -> Tree: ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version in pypi is more strict:

>>> parser.parse(source=print)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: First argument to parse must be bytes
>>> parser.parse(source=bytearray(b'asd'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: First argument to parse must be bytes

def current_field_name(self) -> None | Incomplete: ...
def goto_first_child(self) -> bool: ...
def goto_next_sibling(self) -> bool: ...
def goto_parent(self) -> bool: ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def goto_parent(self) -> bool: ...
def goto_parent(self) -> bool: ...
def copy(self) -> TreeCursor: ...

One more

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't exist in the released version.

@github-actions

This comment has been minimized.

6 similar comments
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

porcupine (https://github.com/Akuli/porcupine)
+ porcupine/plugins/highlight/tree_sitter_highlighter.py:20: error: Unused "type: ignore" comment

@JelleZijlstra JelleZijlstra merged commit c9ebeb5 into python:master Aug 17, 2022
@Akuli Akuli deleted the ts branch August 17, 2022 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants