rustdoc-search: yet another stringdex optimization attempt #145911

notriddle · 2025-08-27T03:00:59Z

This one's uses a different tactic. It shouldn't significantly increase the amount of downloaded index data, but still reduces the amount of disk usage.

This one works by changing the suffix-only node representation to omit some data that's needed for checking. Since those nodes make up the bulk of the tree, it reduces the data they store, but also requires validating the match by fetching the name itself (but the names list is pretty small, and when I tried it with wordnet "indexing" it was about the same).

r? @GuillaumeGomez

rustbot · 2025-08-27T03:01:06Z

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

Some changes occurred in HTML/CSS/JS.

cc @GuillaumeGomez, @jsha, @lolbinarycat

notriddle · 2025-08-27T04:04:16Z

@bors try @rust-timer queue

rustdoc-search: yet another stringdex optimization attempt

klensy · 2025-08-27T05:53:04Z

N.B. pin stringdex version when updated next time, so it won't fail weekly dep updates.

rust-bors · 2025-08-27T06:21:28Z

☀️ Try build successful (CI)
Build commit: b96101a (b96101aded737860cab9b6e77634f4563f5100cb, parent: 176d8dbce6a7a7a4541d1c980d4a85dfb76ce1f1)

rust-timer · 2025-08-27T08:25:45Z

Finished benchmarking commit (b96101a): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	2.9%	[2.9%, 2.9%]	1
Regressions ❌ (secondary)	1.2%	[1.2%, 1.2%]	1
Improvements ✅ (primary)	-1.1%	[-1.1%, -1.1%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.9%	[-1.1%, 2.9%]	2

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

Results (secondary 3.8%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.8%	[3.8%, 3.8%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

Results (secondary -0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.0%	[-0.0%, -0.0%]	1
All ❌✅ (primary)	-	-	0

Bootstrap: 467.709s -> 467.04s (-0.14%)
Artifact size: 391.16 MiB -> 391.21 MiB (0.01%)

notriddle · 2025-08-28T22:39:24Z

@bors try @rust-timer queue

rustdoc-search: yet another stringdex optimization attempt

rust-bors · 2025-08-29T00:56:06Z

☀️ Try build successful (CI)
Build commit: 8689c32 (8689c329d651928f18c308f946f14f54995111a6, parent: 35d55b34bffd51384ac430cc20852b7d16dd5a90)

rust-timer · 2025-08-29T02:05:24Z

Finished benchmarking commit (8689c32): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	1.1%	[0.1%, 3.1%]	3
Regressions ❌ (secondary)	0.4%	[0.1%, 0.8%]	3
Improvements ✅ (primary)	-1.2%	[-1.2%, -1.2%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.5%	[-1.2%, 3.1%]	4

Max RSS (memory usage)

Results (primary -2.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.4%	[-2.4%, -2.4%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-2.4%	[-2.4%, -2.4%]	1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 466.374s -> 466.216s (-0.03%)
Artifact size: 388.50 MiB -> 388.54 MiB (0.01%)

This one's uses a different tactic. It shouldn't significantly increase the amount of downloaded index data, but still reduces the amount of disk usage. This one works by changing the suffix-only node representation to omit some data that's needed for checking. Since those nodes make up the bulk of the tree, it reduces the data they store, but also requires validating the match by fetching the name itself (but the names list is pretty small, and when I tried it with wordnet "indexing" it was about the same).

notriddle · 2025-09-04T00:10:54Z

@bors try @rust-timer queue

rustdoc-search: yet another stringdex optimization attempt

rust-bors · 2025-09-04T02:31:29Z

☀️ Try build successful (CI)
Build commit: a9bbf71 (a9bbf71c81c38ab2f1121f4e975e0ff01f88f8c7, parent: a1208bf765ba783ee4ebdc4c29ab0a0c215806ef)

rust-timer · 2025-09-04T04:47:53Z

Finished benchmarking commit (a9bbf71): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.8%	[-6.1%, -0.1%]	11
Improvements ✅ (secondary)	-0.7%	[-2.4%, -0.1%]	4
All ❌✅ (primary)	-0.8%	[-6.1%, -0.1%]	11

Max RSS (memory usage)

Results (primary -3.0%, secondary 3.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	4.5%	[4.5%, 4.5%]	1
Regressions ❌ (secondary)	3.2%	[3.2%, 3.2%]	1
Improvements ✅ (primary)	-10.5%	[-10.5%, -10.5%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-3.0%	[-10.5%, 4.5%]	2

Cycles

Results (primary -4.0%, secondary 2.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.0%	[2.0%, 2.0%]	1
Improvements ✅ (primary)	-4.0%	[-4.0%, -4.0%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-4.0%	[-4.0%, -4.0%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 466.26s -> 465.954s (-0.07%)
Artifact size: 388.41 MiB -> 388.35 MiB (-0.02%)

GuillaumeGomez · 2025-09-04T12:13:42Z

Looks good to me, thanks! r=me once you think it's ready.

Also, we should make add some checks to see how many files the rustdoc search generates. And checks downloads and the total size of the downloaded files for a given search to have a broad idea of how things evolve.

notriddle · 2025-09-04T15:48:07Z

Doesn't https://perf.rust-lang.org/compare.html?start=a1208bf765ba783ee4ebdc4c29ab0a0c215806ef&end=a9bbf71c81c38ab2f1121f4e975e0ff01f88f8c7&stat=size%3Adoc_files_count already show the total number of generated files?

Anyway,

@bors r=GuillaumeGomez rollup=never

bors · 2025-09-04T15:48:10Z

📌 Commit 80e1805 has been approved by GuillaumeGomez

It is now in the queue for this repository.

bors · 2025-09-04T19:40:43Z

⌛ Testing commit 80e1805 with merge af00ff2...

bors · 2025-09-04T22:50:26Z

☀️ Test successful - checks-actions
Approved by: GuillaumeGomez
Pushing af00ff2 to master...

github-actions · 2025-09-04T22:53:33Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 0d6a806 (parent) -> af00ff2 (this PR)

Test differences

Show 3 test diffs

3 doctest diffs were found. These are ignored, as they are noisy.

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard af00ff2ce62b6617ed19305ae39e135ac71d0b22 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

dist-aarch64-linux: 6344.7s -> 8784.2s (38.5%)
dist-aarch64-apple: 8323.7s -> 7061.4s (-15.2%)
dist-apple-various: 4524.3s -> 3866.7s (-14.5%)
aarch64-gnu-debug: 3731.6s -> 4069.9s (9.1%)
dist-ohos-x86_64: 4141.9s -> 4446.3s (7.3%)
tidy: 200.6s -> 185.9s (-7.3%)
x86_64-gnu-stable: 6832.1s -> 7254.3s (6.2%)
aarch64-gnu-llvm-19-1: 3381.6s -> 3173.7s (-6.1%)
aarch64-msvc-2: 4769.3s -> 5020.3s (5.3%)
x86_64-rust-for-linux: 2506.5s -> 2634.0s (5.1%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-09-05T00:02:46Z

Finished benchmarking commit (af00ff2): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.8%	[-6.0%, -0.1%]	11
Improvements ✅ (secondary)	-1.0%	[-2.4%, -0.2%]	3
All ❌✅ (primary)	-0.8%	[-6.0%, -0.1%]	11

Max RSS (memory usage)

Results (secondary 1.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.1%	[2.0%, 6.6%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.9%	[-4.0%, -1.9%]	2
All ❌✅ (primary)	-	-	0

Cycles

Results (primary -4.9%, secondary -3.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-4.9%	[-4.9%, -4.9%]	1
Improvements ✅ (secondary)	-3.6%	[-5.2%, -2.1%]	2
All ❌✅ (primary)	-4.9%	[-4.9%, -4.9%]	1