Search for code lines including brackets + search on different branches

[hnfhtw]

Dear Sourcebot Team,

thank you for this great tool. I am currently evaluating it for a self-hosted Gitlab environment and it works well. I run the docker container in WSL and needed to apply this workaround here to get it running: #263 (comment)

May I ask two questions:

1. I can search for content (e.g. function names) but as soon as I add brackets the search doesn't work - is there an escape character to include brackets? I want to search e.g. for occurrences of "myFunction(argument1, argument2);"

2. It looks like Sourcebot clones all the repos that I have configured and then searches in the default branch. Is there a possibility to search in ALL branches of ALL repos?

Thanks in Advance,
Harald

[brendan-kellam]

thanks for using the tool!

Answering your questions:

1. The TLDR answer is that you can use \ to escape brackets. For example, foo\(bar\);. If your search includes spaces, you should use wrap the expression in "" and use \\ (since \ now needs to be escaped). E.g., "myFunction\\(argument1, argument2\\);". Checkout this doc for a quick reference.

   The more in-depth answer is that queries consist of space separated regular expressions that are AND'd together on a per-file basis (e.g., foo bar means that the regex /foo/ and the regex /bar/ must both match in a given file for that file to be included in the search results). The or keyword allows you to change this behaviour s.t., regexs are OR'd together (foo or bar means that either /foo/ or /bar (or both) must match in a given file).

   Parenthesis () allow you to group expressions together s.t., more complex boolean expressions can be formed (e.g., foo (bar or baz) matches files with regex /foo/ and either /bar/ or /baz/). To escape (), backslashes \ are used (e.g., foo\(bar\) matches files with regex /foo\(bar\)/).

   Wrapping an expression with quotes "" allows you to include spaces in that expression (e.g., "foo bar" matches regex /foo bar/). Any parenthesis will still group expressions inside of quotes (maybe this should not be the case..). To escape them, two backlashes are used \\ (e.g., "foo\\(bar, baz\\)" matches regex /foo\(bar, baz\)/). A quirk of our syntax is that to escape quotes inside of a expression wrapped in quotes requires a single backslash (e.g., "foo \\(\"this is a string\"\\)" matches regex /foo\("this is a string"\)/).

   Phew that was a long-winded explanation 😅. I think there is allot of work that we can do to make this all more intuitive (e.g., turn of regexs & expression groups by default, add error squiggles to the query box when a query is malformed, better docs etc.).

2. The tool can do that, but it's not very well supported (see this issue). Could you explain what your use-case is for "all branch x all repo" searches? We are planning on over-halling how we approach multi-branch search, so your input would help us support use-cases like yours.

[hnfhtw]

Hi Brendan, thank you for your response and sorry for my late answer. Searching including brackets now works well. Our use case (for search in multiple branches) is the following:

We have some repos where one repo hosts e.g. 2 or 3 different applications/projects. It might be that one application lives in repo A on branch X, and another application in the same repo A on branch Y. The default branch of that repo is e.g. branch X -> so by default we would not find code in branch Y.

Our main use case for sourcebot is to search for files and faulty code in all repos. E.g. if I find a problem with some code lines in a project A I want to see which other projects have the same faulty code lines (and need to be fixed as well).

[brendan-kellam]

Ok gotcha, that makes sense - I think the implementation as it is should be able to support this scenario (**assuming any given repo has 64 or less branches - zoekt has a limitation here).

You could structure your config something like the following:

{
    "$schema": "./schemas/v3/index.json",
    "connections": {
        // Index branches X and Y (and HEAD) of repo_a.
        "repo_a_connection": {
            "type": "github",
            "repos": [
                "org/repo_a"
            ],
            "revisions": {
                "branches": [
                    "X",
                    "Y"
                ]
            }
        },

        // Index _all_ branches of repo_b
        "repo_b_connection": {
            "type": "github",
            "repos": [
                "org/repo_b"
            ],
            "revisions": {
                // Glob pattern helps us here
                "branches": [
                    "**"
                ]
            }
        },

        // Index every other repo other than repo_a and repo_b.
        // Only HEAD is indexed.
        "everything_else_connection": {
            "type": "github",
            "repos": [
                // ...
            ],
            "exclude": {
                "repos": [
                    "org/repo_a",
                    "org/repo_b"
                ]
            }
        }
    }
}

I just put up a PR in #281 to change the "search all branches" syntax a bit.. so after that merges, you will be able to do queries like:

"error message" rev:* - searches "error message" across all repos & all branches.