Slow builds for larger resources embedded in code

[nixberg]

Description

As suggested in the PR, Resource.embedInCode(_:) appears to be inefficient for larger resources.
For a minimal .executableTarget with a 165 KB resource to be embedded, the debug build takes some 200 seconds on my machine. 19 seconds for the release build.

Expected behavior

No response

Actual behavior

No response

Steps to reproduce

No response

Swift Package Manager version/commit hash

No response

Swift & OS version (output of swift --version ; uname -a)

No response

[neonichu]

I hadn't even thought about compilation speed tbh. I wonder whether #68141 makes this better since we essentially generate a large array.

[nixberg]

FWIW. Somewhat subquadratic in debug mode, roughly linear in release mode.

Debug:
 25K:   7.15s
 50K:  21.65s
100K:  77.35s
200K: 300.21s

Release:
 25K:   3.33s
 50K:   5.86s
100K:  11.31s
200K:  23.20s

[Lancelotbronner]

I had a 1.2 MB JSON file (https://lua-api.factorio.com/latest/prototype-api.json), please note that embedInCode is unusable for this.

[migueldeicaza]

I had a 9 megabyte file, the compiler aborts due to the type check taking too long.

The generated file is a 30 megabyte swift file.

[MaxDesiatov]

If type checking is taking too long, I don't think there's anything specific to the package manager here. Moving the issue accordingly.

[Lancelotbronner]

I read somewhere that clang implemented the C #embed using a special builtin.

Could Swift also do the same thing? Turning a 9 MB resource into a 30 MB generated file that then needs to be parsed and type-checked doesn't seem ideal.

[migueldeicaza]

Rather than introduce a new language extension, the compiler could just flag during parsing "I have seen that all of these are byte constants" as an annotation, and proceed without the type resolver.

[DagAgren]

Is there any progress on this? I keep trying to use this feature only to find out it's basically unusable. I really want to use this!