Skip to content

✨ Analyser CLI #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 11, 2025
Merged

✨ Analyser CLI #11

merged 1 commit into from
Aug 11, 2025

Conversation

juiwenchen
Copy link
Contributor

@juiwenchen juiwenchen commented Aug 1, 2025

Add Analyzer :

Impemented Anaylzer CLI and API

  1. Three types of markers extraction from comments:
  • one-line needs declared by one-line comment style
  • Need ID Refs
  • Marked RST block
  1. Extract the scopes such Class and Functions tagged by these markers

  2. CLI can only be used with TOML config file

Remove Virtual Docs and integrate one-line parser into Analyzer

  1. One-line comment pattern is integrated into Analyzer as one of the marker type one-line needs

  2. caching in Virtual Docs is removed and will bring back in the future

Adaption of Sphinx directive src-trace

  1. using Analyzer to extract One-line needs

  2. Config files of src-trace and Anaylzer CLI are not compatible and will adapt in the future

@juiwenchen juiwenchen force-pushed the analyser-cli branch 2 times, most recently from 1d8c47c to 2ccc027 Compare August 5, 2025 08:42
@juiwenchen juiwenchen changed the title Analyser cli ✨ Analyser CLI Aug 5, 2025
@juiwenchen juiwenchen requested a review from ubmarco August 5, 2025 12:00
@ubmarco
Copy link
Member

ubmarco commented Aug 5, 2025

Default for marker should be @need-ids:.
This should be documented. It will avoid the following surprise:

    "marker": "@",
    "need_ids": ["req-id:", "need_001", "need_002", "need_003", "need_004"]
  },

while it should be

    "marker": "@",
    "type": "need-id-refs",
    "need_ids": ["need_001", "need_002", "need_003", "need_004"]
  },

There a couple of use case of the analyzer and the downstream consumers do not know what use case is extracted.
Add this:

"type": "need-id-refs",

Use cases:

  • need-id-refs
  • need
  • rst

An example output json can look like:

[
  {
    "filepath": "tests/data/anchors/dummy_1.cpp",
    "remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/f2d8d98ca10361663191c4c96255f39db9b5f518/tests/data/anchors/dummy_1.cpp#L3",
    "source_map": {
        "start": {
            "row": 10,
            "column": 3
        },
        "end": {
            "row": 13,
            "column": 16
        }
    },
    "marker": "@req-id:",
    "type": "need-id-refs",
    "need_ids": ["need_001", "need_002", "need_003", "need_004"]
  },
  {
    "filepath": "tests/data/anchors/dummy_1.cpp",
    "remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/f2d8d98ca10361663191c4c96255f39db9b5f518/tests/data/anchors/dummy_1.cpp#L3",
    "source_map": {
        "start": {
            "row": 20,
            "column": 3
        },
        "end": {
            "row": 26,
            "column": 15
        }
    },
    "type": "need",
    "need": {
        "id": "TEST_001",
        "type": "test",
        "title": "Test a function"
    }
  },
  {
    "filepath": "tests/data/anchors/dummy_1.cpp",
    "remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/f2d8d98ca10361663191c4c96255f39db9b5f518/tests/data/anchors/dummy_1.cpp#L3",
    "source_map": {
        "start": {
            "row": 30,
            "column": 3
        },
        "end": {
            "row": 35,
            "column": 15
        }
    },
    "type": "rst",
    "rst": "some paragraph with **bold** text and a \n\n.. test:: Test case\n   :id: TEST_002\n   This is a test case for the function.\n"
  }
]

pyproject.toml Outdated
"tree-sitter-python>=0.23.6",
# Compiling issue on manylinux aarch64 image, so install from git instead of Pypi
# https://github.com/tree-sitter/py-tree-sitter/issues/386#issuecomment-3101430799
"tree-sitter @ git+https://github.com/tree-sitter/py-tree-sitter",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unpinned dependency, either pick a release or refer to a commit, document that clang is required due to this bug, also point out what's wrong.

@juiwenchen juiwenchen requested a review from ubmarco August 11, 2025 13:41
@juiwenchen juiwenchen force-pushed the analyser-cli branch 2 times, most recently from 11f83b5 to ee18aa3 Compare August 11, 2025 16:19
Copy link
Member

@ubmarco ubmarco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did an online review, looks good for the first version

- integated vdocs to analyse
- support three types of markers
- export json for other applications
- support CLI and API
@juiwenchen juiwenchen merged commit 481a69e into main Aug 11, 2025
8 checks passed
@juiwenchen juiwenchen deleted the analyser-cli branch August 11, 2025 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants