Extremely long compile times when compiling examples/sync.rs with nightly compiler #109

earthnuker · 2021-11-27T01:23:25Z

Hello everyone,

after copying the content of examples/sync.rs into an empty rust project and adding the following versions of the dependencies:

[dependencies]
anyhow = "1.0.48"
async-std = { version = "1.10.0", features = ["attributes"] }
futures = "0.3.18"
ipfs-embed = "0.22.4"
libipld = "0.12.0"
rand = "0.8.4"
tracing-subscriber = {version="0.3.2", features=["env-filter"]}

when trying to compile it with either the latest stable (rustc 1.56.1 (59eed8a2a 2021-11-01)) or nightly (rustc 1.58.0-nightly (dd549dcab 2021-11-25)) nightly-x86_64-pc-windows-msvc toolchain on Windows 10 i get compile time exceeding 2 hours, i can provide a verbose compiler log if required.

Best regards,

Earthnuker

The text was updated successfully, but these errors were encountered:

dvc94ch · 2021-11-27T10:19:47Z

Do you have insane compile times compiling other rust projects? What's the CPU/mem utilisation during that time?

dvc94ch · 2021-11-27T10:25:27Z

I'll guess you have a very low powered device and 90% of your CPU/mem is being used by windows to monitor you and send a real time voice/video feed to the central machine learning center.

earthnuker · 2021-11-27T14:28:23Z

Hello, thanks for the quick reply,

i just woke up and it did finish compiling after 6 hours and 40 minutes, but when i try to run it the executable overflows its stack and crashes.

Do you have insane compile times compiling other rust projects? What's the CPU/mem utilisation during that time?

no, this is the only dependency that has caused the issue so far.

my CPU is a Intel i7-4770K (8) @ 3.5GHz, 32 GB of DDR3 RAM

rustc uses one CPU core and about 750MB of memory during the phase where it is slow

I'll guess you have a very low powered device and 90% of your CPU/mem is being used by windows to monitor you and send a real time voice/video feed to the central machine learning center.

the same behavior occurs on a Linux machine with a Intel i5-2400 (4) @ 3.400GHz, 20 GB of RAM, and don't you think the "haha windows 10 is spying on you all the time" joke is getting a bit old?

Best regards,

Earthnuker

dvc94ch · 2021-11-27T16:29:29Z

on a fresh checkout it builds for me in a bit over a minute.

time cargo build
________________________________________________________
Executed in   87.85 secs    fish           external
   usr time  566.68 secs   65.78 millis  566.61 secs
   sys time   48.83 secs    6.16 millis   48.82 secs

after that I built the examples

time cargo build --examples
________________________________________________________
Executed in   34.02 secs    fish           external
   usr time  149.15 secs    0.00 micros  149.15 secs
   sys time   11.37 secs  746.00 micros   11.37 secs

don't you think the "haha windows 10 is spying on you all the time" joke is getting a bit old?

well, I doubt android, ios or macos are much better. why do you think amazon built alexa if not to spy on you?

earthnuker · 2021-11-27T17:03:44Z

on a fresh checkout it builds for me in a bit over a minute.
time cargo build
________________________________________________________
Executed in   87.85 secs    fish           external
   usr time  566.68 secs   65.78 millis  566.61 secs
   sys time   48.83 secs    6.16 millis   48.82 secs
after that I built the examples
time cargo build --examples
________________________________________________________
Executed in   34.02 secs    fish           external
   usr time  149.15 secs    0.00 micros  149.15 secs
   sys time   11.37 secs  746.00 micros   11.37 secs
don't you think the "haha windows 10 is spying on you all the time" joke is getting a bit old?

well, I doubt android, ios or macos are much better. why do you think amazon built alexa if not to spy on you?

on a fresh checkout

cargo build runs without issue in about 2 minutes
cargo build --examples still hangs

i tried switching to the latest nightly-x86_64-pc-windows-gnu toolchain, same result

cryptoquick · 2021-11-27T20:26:06Z

@earthnuker, you can try cross-compiling from linux or WSL:

https://github.com/rust-embedded/cross

Also, @dvc94ch, you gotta remember, not everybody is uncomfortable being spied on. Some people don't care how much corporations are being paid to manipulate them as long as they're handing out free stuff.

dvc94ch · 2021-11-27T21:03:28Z

We can probably leave the spying stuff, it is admittedly off topic. You could try using the lld linker. Linking takes a lot of ram, I assume that's where it's hanging.

…

On Sat, Nov 27, 2021, 21:26 Hunter Trujillo ***@***.***> wrote: @earthnuker <https://github.com/Earthnuker>, you can try cross-compiling from linux or WSL: https://github.com/rust-embedded/cross Also, @dvc94ch <https://github.com/dvc94ch>, you gotta remember, not everybody is uncomfortable being spied on. Some people don't care how much corporations are being paid to manipulate them as long as they're handing out free stuff. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#109 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFVDL7TYP5ZF52MF2SI6GDUOE5GRANCNFSM5I3THPXQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

earthnuker · 2021-11-27T22:11:56Z

@earthnuker, you can try cross-compiling from linux or WSL:

https://github.com/rust-embedded/cross

Also, @dvc94ch, you gotta remember, not everybody is uncomfortable being spied on. Some people don't care how much corporations are being paid to manipulate them as long as they're handing out free stuff.

using a cross compiler does not solve the issue

We can probably leave the spying stuff, it is admittedly off topic. You could try using the lld linker. Linking takes a lot of ram, I assume that's where it's hanging.

as i said before, this machine has 32 GB of memory, the server i'm compiling on has 20 GB of memory, and the rust compiler barely uses a gigabyte of memory

cryptoquick · 2021-11-27T23:46:40Z

@earthnuker if you can use WSL to cross compile the program, and it can run on Windows, I think that's a reasonable workaround.

earthnuker · 2021-11-27T23:53:18Z

@earthnuker if you can use WSL to cross compile the program, and it can run on Windows, I think that's a reasonable workaround.

thing i tried so far:

nightly and stable compiler natively on windows 10 (both msvc and gnu targets): hangs
nightly and stable compiler natively on linux: hangs
nightly compiler targeting x86_64-pc-windows-gnu through cross running on a linux machine: hangs

why would using WSL fix the issue?

update: compiling in WSL also hangs at Building [=======================> ] 410/411: ipfs(bin) (ipfs is the name of my test crate containing the sync.rs example code)

dvc94ch · 2021-11-28T01:09:33Z

don't know what to tell you other than maybe try a different linker

earthnuker · 2021-11-28T01:35:30Z

tried with

[target.x86_64-pc-windows-msvc]
rustflags = [
    "-C","linker-flavor=lld-link"
]

no change

update: did some more testing, stripped out all dependencies that are not needed, so now my Cargo.toml is

[package]
name = "ipfs_test"
version = "0.1.0"
edition = "2021"

[dependencies]
async-std = { version = "1.10.0", features = ["attributes"] }
ipfs-embed = "0.22.4"
libipld = "0.12.0"

and my code is

use ipfs_embed::{generate_keypair, Config, Ipfs};
use libipld::store::DefaultParams;

#[async_std::main]
async fn main() {
    let mut config = Config::new("out".as_ref(), generate_keypair());
    config.network.kad = None;
    let a = Ipfs::<DefaultParams>::new(config).await.unwrap();
}

that seems to be the minimum necesarry to reproduce the issue for me, as soon as i remove the line that calls the new method of the Ipfs struct it compiles fine

wngr · 2021-11-28T18:35:08Z

Running your sample on Linux (target x86_64-unknown-linux-gnu) works as expected:

✗ time cargo build
// ...
cargo build  675.98s user 52.79s system 634% cpu 1:54.87 total
✗ rustc --version
rustc 1.56.1 (59eed8a2a 2021-11-01)

Maybe you have some global .cargo settings or something? I guess this can only be debugged if you're able to reproduce this on a vanilla host, maybe a public github repo using github actions?

Edit: Added said repo at https://github.com/wngr/test-109, CI job works even w/ x86_64-pc-windows-msvc.

earthnuker · 2021-11-28T20:56:43Z

did some more testing:

clean docker container using https://github.com/Earthnuker/ipfs_embed_109 and docker build .

tested on:

github CI
the linux machine with 20 GB of RAM, hangs
a VM on my brother's VM host, hangs, killed after ~20 minutes at Compiling ipfs_test
a friend's Linux machine, hangs, killed after ~20 minutes at Compiling ipfs_test

another friend's Linux machine, hangs, killed after ~20 minutes at Compiling ipfs_test

perf trace has the following to say:

204698.147 (500.112 ms): futex(uaddr: 0x55e1db6c4bcc, op: WAIT|PRIVATE_FLAG, utime: 0x7fff7e6b3ed0) = -1 ETIMEDOUT (Connection timed out)
205198.273 ( 0.002 ms): futex(uaddr: 0x55e1db6c5590, op: WAKE|PRIVATE_FLAG, val: 1)           = 0
205198.286 (500.083 ms): futex(uaddr: 0x55e1db6c4bcc, op: WAIT|PRIVATE_FLAG, utime: 0x7fff7e6b3ed0) = -1 ETIMEDOUT (Connection timed out)
205698.382 ( 0.001 ms): futex(uaddr: 0x55e1db6c5590, op: WAKE|PRIVATE_FLAG, val: 1)           = 0
205698.395 (500.084 ms): futex(uaddr: 0x55e1db6c4bcc, op: WAIT|PRIVATE_FLAG, utime: 0x7fff7e6b3ed0) = -1 ETIMEDOUT (Connection timed out)
206198.490 ( 0.001 ms): futex(uaddr: 0x55e1db6c5590, op: WAKE|PRIVATE_FLAG, val: 1)           = 0
206198.504 (500.113 ms): futex(uaddr: 0x55e1db6c4bcc, op: WAIT|PRIVATE_FLAG, utime: 0x7fff7e6b3ed0) = -1 ETIMEDOUT (Connection timed out)
206698.628 ( 0.001 ms): futex(uaddr: 0x55e1db6c5590, op: WAKE|PRIVATE_FLAG, val: 1)           = 0
206698.642 (500.061 ms): futex(uaddr: 0x55e1db6c4bcc, op: WAIT|PRIVATE_FLAG, utime: 0x7fff7e6b3ed0) = -1 ETIMEDOUT (Connection timed out)
207198.713 ( 0.001 ms): futex(uaddr: 0x55e1db6c5590, op: WAKE|PRIVATE_FLAG, val: 1)           = 0
207198.726 (500.125 ms): futex(uaddr: 0x55e1db6c4bcc, op: WAIT|PRIVATE_FLAG, utime: 0x7fff7e6b3ed0) = -1 ETIMEDOUT (Connection timed out)
207698.864 ( 0.001 ms): futex(uaddr: 0x55e1db6c5590, op: WAKE|PRIVATE_FLAG, val: 1)           = 0
207698.880 (500.074 ms): futex(uaddr: 0x55e1db6c4bcc, op: WAIT|PRIVATE_FLAG, utime: 0x7fff7e6b3ed0) = -1 ETIMEDOUT (Connection timed out)
208198.964 ( 0.001 ms): futex(uaddr: 0x55e1db6c5590, op: WAKE|PRIVATE_FLAG, val: 1)           = 0

i also i pushed a second commit to the test repo that removes the codegen-units = 1 options from Cargo.toml, although that has made no difference on my windows machine

earthnuker · 2021-11-29T01:36:49Z

the github CI job with codegen-units=1 finished after 4 hours and 20 minutes, the job without codegen-units=1 is still running, clocking in at over 5 hours now and will likely be killed by a timeout before it finishes.

wngr · 2021-11-29T10:10:45Z

So I tried your reproducer, which also hangs for me using nightly. However, with stable everything completes in a timely manner. Great, if you could find the issue causing this. But given that this is only reproducible using the nightly compiler, I suggest to wait for the next stable Rust release.

earthnuker · 2021-11-29T12:00:19Z

just tried it with a stable compiler, compiles fine, still crashes with a stack overflow when trying to execute on windows, runs fine on linux

rishflab · 2021-12-28T04:12:32Z

I have the same compile time issue when running on 1.57-x86_64-unknown-linux-gnu. This issue is solved by downgrading to 1.56.1-x86_64-unknown-linux-gnu.

rkuhn · 2021-12-28T11:39:56Z

I’m also hitting this on 1.57, compiler phase monomorphization_collector_graph_walk doesn’t finish before I feel compelled to kill it. Unfortunately there isn’t much literature on how to find the culprit …

rkuhn · 2021-12-28T11:51:48Z

There’s a rustc bug in the vicinity of “deeply nested decorator types” — libp2p definitely loves those.

rkuhn · 2022-10-18T10:02:14Z

fixed in Rust 1.59

earthnuker changed the title ~~Extremely long compile times when compiling examples/sync.rs~~ Extremely long compile times when compiling examples/sync.rs with nightly compiler Nov 29, 2021

rkuhn mentioned this issue Dec 28, 2021

Compile-time regression between 1.56.1 and 1.57.0 for deeply nested decorator types rust-lang/rust#91598

Closed

rkuhn closed this as completed Oct 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extremely long compile times when compiling examples/sync.rs with nightly compiler #109

Extremely long compile times when compiling examples/sync.rs with nightly compiler #109

earthnuker commented Nov 27, 2021

dvc94ch commented Nov 27, 2021

dvc94ch commented Nov 27, 2021

earthnuker commented Nov 27, 2021 •

edited

Loading

dvc94ch commented Nov 27, 2021

earthnuker commented Nov 27, 2021

cryptoquick commented Nov 27, 2021

dvc94ch commented Nov 27, 2021 via email

earthnuker commented Nov 27, 2021

cryptoquick commented Nov 27, 2021

earthnuker commented Nov 27, 2021 •

edited

Loading

dvc94ch commented Nov 28, 2021

earthnuker commented Nov 28, 2021 •

edited

Loading

wngr commented Nov 28, 2021 •

edited

Loading

earthnuker commented Nov 28, 2021

earthnuker commented Nov 29, 2021

wngr commented Nov 29, 2021

earthnuker commented Nov 29, 2021

rishflab commented Dec 28, 2021 •

edited

Loading

rkuhn commented Dec 28, 2021

rkuhn commented Dec 28, 2021

rkuhn commented Oct 18, 2022

Extremely long compile times when compiling examples/sync.rs with nightly compiler #109

Extremely long compile times when compiling examples/sync.rs with nightly compiler #109

Comments

earthnuker commented Nov 27, 2021

dvc94ch commented Nov 27, 2021

dvc94ch commented Nov 27, 2021

earthnuker commented Nov 27, 2021 • edited Loading

dvc94ch commented Nov 27, 2021

earthnuker commented Nov 27, 2021

cryptoquick commented Nov 27, 2021

dvc94ch commented Nov 27, 2021 via email

earthnuker commented Nov 27, 2021

cryptoquick commented Nov 27, 2021

earthnuker commented Nov 27, 2021 • edited Loading

dvc94ch commented Nov 28, 2021

earthnuker commented Nov 28, 2021 • edited Loading

wngr commented Nov 28, 2021 • edited Loading

earthnuker commented Nov 28, 2021

earthnuker commented Nov 29, 2021

wngr commented Nov 29, 2021

earthnuker commented Nov 29, 2021

rishflab commented Dec 28, 2021 • edited Loading

rkuhn commented Dec 28, 2021

rkuhn commented Dec 28, 2021

rkuhn commented Oct 18, 2022

earthnuker commented Nov 27, 2021 •

edited

Loading

earthnuker commented Nov 27, 2021 •

edited

Loading

earthnuker commented Nov 28, 2021 •

edited

Loading

wngr commented Nov 28, 2021 •

edited

Loading

rishflab commented Dec 28, 2021 •

edited

Loading