New language implementation reference (mozilla#692)

Co-authored-by: Marco Castelluccio <[email protected]>

Author: dburriss

rust-code-analysis-book/src/SUMMARY.md

@@ -8,3 +8,5 @@
     - [Nodes](commands/nodes.md)
     - [Rest API](commands/rest.md)
 - [Developers Guide](developers/README.md)
+    - [How-to: Add a new language](developers/new-language.md)
+    - [How-to: Implement LoC](developers/loc.md)

rust-code-analysis-book/src/developers/loc.md

@@ -0,0 +1,57 @@
+# Lines of Code (LoC)
+
+In this document we give some guidance on how to implement the LoC metrics available in this crate.
+[Lines of code](https://en.wikipedia.org/wiki/Source_lines_of_code) is a software metric that gives an indication of the size of some source code by counting the lines of the source code.
+There are many types of LoC so we will first explain those by way of an example.
+
+## Types of LoC
+
+```rust
+/*
+Instruction: Implement factorial function
+For extra credits, do not use mutable state or a imperative loop like `for` or `while`.
+ */
+
+/// Factorial: n! = n*(n-1)*(n-2)*(n-3)...3*2*1
+fn factorial(num: u64) -> u64 {
+    
+    // use `product` on `Iterator`
+    (1..=num).product()
+}
+```
+
+The example above will be used to illustrate each of the **LoC** metrics described below.
+
+### SLOC
+
+A straight count of all lines in the file including code, comments, and blank lines.  
+METRIC VALUE: 11
+
+### PLOC
+
+A count of the instruction lines of code contained in the source code. This would include any brackets or similar syntax on a new line. 
+Note that comments and blank lines are not counted in this.   
+METRIC VALUE: 3
+
+### LLOC
+
+The "logical" lines is a count of the number of statements in the code. Note that what a statement is depends on the language.  
+In the above example there is only a single statement which id the function call of `product` with the `Iterator` as its argument.  
+METRIC VALUE: 1
+
+### CLOC
+
+A count of the comments in the code. The type of comment does not matter ie single line, block, or doc.  
+METRIC VALUE: 6
+
+### BLANK
+
+Last but not least, this metric counts the blank lines present in a code.
+METRIC VALUE: 2
+
+## Implementation
+
+To implement the LoC related metrics described above you need to implement the `Loc` trait for the language you want to support.
+
+This requires implementing the `compute` function.
+See [/src/metrics/loc.rs](https://github.com/mozilla/rust-code-analysis/blob/master/src/metrics/loc.rs) for where to implement, as well as examples from other languages.

rust-code-analysis-book/src/developers/new-language.md

@@ -0,0 +1,56 @@
+# Supporting a new language
+
+This section is to help developers implement support for a new language in `rust-code-analysis`.
+
+To implement a new language, two steps are required:
+
+1. Generate the grammar
+2. Add the grammar to `rust-code-analysis`
+
+A number of [metrics are supported](https://mozilla.github.io/rust-code-analysis/metrics.html) and help to implement those are covered elsewhere in the documentation.
+
+## Generating the grammar
+
+As a **prerequisite** for adding a new grammar, there needs to exist a [tree-sitter](https://github.com/tree-sitter) version for the desired language that matches the [version used in this project](https://github.com/mozilla/rust-code-analysis/blob/master/Cargo.toml).
+
+The grammars are generated by a project in this repository called [enums](https://github.com/mozilla/rust-code-analysis/tree/master/enums). The following steps add the language support from the language crate and generate an enum file that is then used as the grammar in this project to evaluate metrics.
+
+1. Add the language specific `tree-sitter` crate to the `enum` crate, making sure to tie it to the `tree-sitter` version used in the `ruse-code-analysis` crate. For example, for the Rust support at time of writing the following line exists in the [/enums/Cargo.toml](https://github.com/mozilla/rust-code-analysis/blob/master/enums/Cargo.toml): `tree-sitter-rust = "version number"`.
+2. Append the language to the `enum` crate in [/enums/src/languages.rs](https://github.com/mozilla/rust-code-analysis/blob/master/enums/src/languages.rs). Keeping with Rust as the example, the line would be `(Rust, tree_sitter_rust)`. The first parameter is the name of the Rust enum that will be generated, the second is the `tree-sitter` function to call to get the language's grammar.
+3. Add a case to the end of the match in `mk_get_language` macro rule in [/enums/src/macros.rs](https://github.com/mozilla/rust-code-analysis/blob/master/enums/src/macros.rs) eg. for Rust `Lang::Rust => tree_sitter_rust::language()`.
+4. Lastly, we execute the [/recreate-grammars.sh](https://github.com/mozilla/rust-code-analysis/blob/master/recreate-grammars.sh) script that runs the `enums` crate to generate the grammar for the new language.
+
+At this point we should have a new grammar file for the new language in [/src/languages/](https://github.com/mozilla/rust-code-analysis/tree/master/src/languages). See [/src/languages/language_rust.rs](https://github.com/mozilla/rust-code-analysis/blob/master/src/languages/language_rust.rs) as an example of the generated enum.
+
+## Adding the new grammar to rust-code-analysis
+
+1. Add the language specific `tree-sitter` crate to the `rust-code-analysis` project, making sure to tie it to the `tree-sitter` version used in this project. For example, for the Rust support at time of writing the following line exists in the [Cargo.toml](https://github.com/mozilla/rust-code-analysis/blob/master/Cargo.toml): `tree-sitter-rust = "0.19.0"`.
+2. Next we add the new `tree-sitter` language namespace to [/src/languages/mod.rs](https://github.com/mozilla/rust-code-analysis/blob/master/src/languages/mod.rs) eg. 
+
+```rust
+pub mod language_rust;
+pub use language_rust::*;
+```
+
+3. Lastly, we add a definition of the language to the arguments of `mk_langs!` macro in [/src/langs.rs](https://github.com/mozilla/rust-code-analysis/blob/master/src/langs.rs).
+
+```rust
+// 1) Name for enum
+// 2) Language description
+// 3) Display name
+// 4) Empty struct name to implement
+// 5) Parser name
+// 6) tree-sitter function to call to get a Language
+// 7) file extensions
+// 8) emacs modes
+(
+    Rust,
+    "The `Rust` language",
+    "rust",
+    RustCode,
+    RustParser,
+    tree_sitter_rust,
+    [rs],
+    ["rust"]
+)
+```

rust-code-analysis-book/src/languages.md

@@ -3,14 +3,16 @@
 This is the list of programming languages parsed by
 **rust-code-analysis**.
 
-* C++
-* C#
-* CSS
-* Go
-* HTML
-* Java
-* JavaScript
-* The JavaScript used in Firefox internal
-* Python
-* Rust
-* Typescript
+- [x] C++
+- [ ] C#
+- [ ] CSS
+- [ ] Go
+- [ ] HTML
+- [ ] Java
+- [x] JavaScript
+- [x] The JavaScript used in Firefox internal
+- [x] Python
+- [x] Rust
+- [x] Typescript
+
+A check indicates which languages have metrics implemented.