Improve code syntax highlighting and formatting in Sphinx

[kinow]

I tried using different styles for pygments, but Shell scripts are not easy to parse visually. I thought it wasn't really good, and @swzCuroverse 's review confirmed that the current output is not good enough.

In the image above, the three command-line commands are being rendered in Sphinx by pygments (MyST vs. rst doesn't really matter here I think). I had to add the $ before the command to distinguish when the code block includes output. But adding prefixes like $ is not considered a good practice as it makes it harder for users to copy-and-paste commands.

This issue is to find a better way to render commands. GitHub produces a good output. It would be good to find one that:

• Has different themes/styles (e.g. in case we have reports from color-blind users, we may need to change the theme, easier if we don't have to change CSS styles).
• Has the option to display line numbers.
• Has the option to display a command prefix.
• Creates an output that is easy to comprehend, especially when you have command-line program and output in the same code block.
• Easy to integrate in Sphinx.

[kinow]

One suggestion from @swzCuroverse: https://github.com/carbon-app/carbon

[kinow]

Getting some good results changing Sphinx settings. Looks like we won't need anything added to the theme 🎉

[kinow]

I was using bash code highlighting, but instead had to use console, Today-I-Learned 😬

Before and after for comparison, updating the pull request soon.

Other improvements:

• Used :caption: and now Sphinx automatically adds a caption above each code block.
• Used :linenos: for line numbering, but it was calling too much attention I think. Can be added later. We can also highlight lines of the code block.
• Added a :name: to each code block, so now readers can link directly to each code block 🙂

Unfortunately the console code block uses the $ prefix to figure out what is a command and what is not (i.e. an output). So we have to keep it. There are other extensions/plug-ins that allow to separate command and output, and even link directly to a line number. But that's for later.

[kinow]

Last thing I'll try after lunch is to make the lexer recognize more Linux commands. It highlights source correctly, but not cwltool, or python.

Glad that @swzCuroverse raised this issue during the review, since I had to modify about 20 code blocks. It would have been a much bigger task if we already had written a lot more sections of the user guide 😵‍💫

[kinow]

Going to leave the extra commands highlighting for another time.

I customized the console lexer to highlight pip, python, cwltool, and other commands, but that doesn't look correct.

from pygments.token import Name
from pygments.lexers.shell import BashSessionLexer

class CwlBashSessionLexer(BashSessionLexer):
    def __init__(self, **options):
        super(CwlBashSessionLexer, self).__init__(**options)
        self._innerLexerCls.tokens['basic'].append(
            (r'\b(pip|python|cwltool|cwl-runner)(?=[\s)`])', Name.Builtin)
        )

lexers['console'] = CwlBashSessionLexer()

The lexer used for the console uses another lexer, the bash lexer. So if, source, for, and other bash language keywords/tokens are parsed and recognized correctly, being also highlighted. But highlighting the command sounds difficult, and not that useful.

At least the new syntax highlighting must make it a lot easier for readers to distinguish commands and output.