Skip to content

Using syntect as a higlighting backend rather than highlight.js #1652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 32 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
8d8c103
Use syntect instead of highlight.js
ThePuzzlemaker Aug 3, 2021
83e4915
Remove profiling release profile
ThePuzzlemaker Aug 3, 2021
1425025
Scope splitting syntect "boring" implementation
notriddle Sep 17, 2021
1fe86a5
Move Playground and comma/space class normalization into a pulldown-c…
notriddle Sep 17, 2021
85d4550
Fix line endings
notriddle Sep 18, 2021
b1e3e56
Detect runs of "boring" lines
notriddle Sep 18, 2021
8266ea9
Add the cache generator subcommand and docs for custom syntaxes
notriddle Sep 18, 2021
cf2b2af
Make the highlighting closer to before
notriddle Sep 18, 2021
5efaec8
Fix build error on no default features
notriddle Sep 18, 2021
67cd22a
Add TOML syntax
notriddle Sep 18, 2021
f38c6e6
Add new syntaxes to guide
notriddle Sep 18, 2021
7feee33
Add background and border around code blocks
notriddle Sep 18, 2021
6075bd1
Add scroll bar for big code blocks
notriddle Sep 18, 2021
924b4bf
Fix incorrectly ordered classes
notriddle Sep 18, 2021
0dd350d
Fix formatting
notriddle Sep 18, 2021
129b560
Put the "syntect" class on the code
notriddle Sep 19, 2021
048042f
fix(docs): added note for where the handlebars syntax came from
notriddle Sep 19, 2021
1de4ab8
fix(init): avoid TOCTTOU bug on syntax CSS dir
notriddle Sep 28, 2021
b42bbbd
Fix broken language in syntaxes README
notriddle Sep 28, 2021
51d8f15
Add syntax-themes.spdx
notriddle Sep 28, 2021
dcf8e3f
Add syntaxes.spdx
notriddle Sep 28, 2021
af917d3
Brighten the blue on tag names in ayu
notriddle Oct 7, 2021
2051b29
Brighten the blue on tag names in ayu
notriddle Oct 7, 2021
d78a75d
Fix compile error
notriddle Oct 7, 2021
9235cbf
Fix console syntax to only see prompts at the beginning of a line
notriddle Oct 7, 2021
abc29e2
Rebase onto current main branch
notriddle Jun 28, 2022
d674758
Merge master updates
KFearsoff Dec 3, 2023
4596901
Update syntect version
KFearsoff Dec 3, 2023
f47f7d6
Merge branch 'master' into temp
KFearsoff Dec 3, 2023
8cab0cb
Fixup documentation
KFearsoff Dec 3, 2023
806d2a9
Merge remote-tracking branch 'origin/master' into notriddle/use-synte…
notriddle Jun 8, 2024
e98349e
Fix test cases
notriddle Jun 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ test_book/book/
# Ignore Jetbrains specific files.
.idea/

src/theme/syntaxes.bin

# Ignore Vim temporary and swap files.
*.sw?
*~
157 changes: 157 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

36 changes: 35 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,20 @@ readme = "README.md"
repository = "https://github.com/rust-lang/mdBook"
description = "Creates a book from markdown files"
rust-version = "1.74"
build = "build.rs"

[build-dependencies.syntect]
version = "5.1.0"
default-features = false
features = [
# regex-onig, the default, isn't portable enough
"regex-fancy",
#"parsing",
"default-syntaxes",
"yaml-load",
"dump-load",
"dump-create",
]

[dependencies]
anyhow = "1.0.71"
Expand Down Expand Up @@ -52,6 +66,21 @@ warp = { version = "0.3.6", default-features = false, features = ["websocket"],
elasticlunr-rs = { version = "3.0.2", optional = true }
ammonia = { version = "4.0.0", optional = true }

[dependencies.syntect]
version = "5.1.0"
default-features = false
features = [
# regex-onig, the default, isn't portable enough
"regex-fancy",
"parsing",
"html",
"default-themes",
"plist-load",
"yaml-load",
"dump-load",
"dump-create",
]

[dev-dependencies]
assert_cmd = "2.0.11"
predicates = "3.0.3"
Expand All @@ -61,10 +90,11 @@ pretty_assertions = "1.3.0"
walkdir = "2.3.3"

[features]
default = ["watch", "serve", "search"]
default = ["watch", "serve", "search", "gen-syntax-cache"]
watch = ["dep:notify", "dep:notify-debouncer-mini", "dep:ignore", "dep:pathdiff", "dep:walkdir"]
serve = ["dep:futures-util", "dep:tokio", "dep:warp"]
search = ["dep:elasticlunr-rs", "dep:ammonia"]
gen-syntax-cache = ["syntect/dump-create"]

[[bin]]
doc = false
Expand All @@ -73,3 +103,7 @@ name = "mdbook"
[[example]]
name = "nop-preprocessor"
test = true

[profile.dev]
opt-level = 1

18 changes: 18 additions & 0 deletions build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
use std::env;
use std::error::Error;
use syntect::dumps::dump_to_file;
use syntect::parsing::SyntaxSet;

pub fn main() -> Result<(), Box<dyn Error>> {
let src_dir = format!(
"{}/src/theme/syntaxes/",
env::var("CARGO_MANIFEST_DIR").unwrap()
);
let dest = format!("{}/syntaxes.bin", env::var("OUT_DIR").unwrap());

let mut builder = SyntaxSet::load_defaults_newlines().into_builder();
builder.add_from_folder(&src_dir, true)?;
dump_to_file(&builder.build(), dest)?;

Ok(())
}
Loading