-
-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Incremental fixes, refactor Zcu.File
#23836
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
In one of my project I currently use non existant modules to have optional depencencies. Simplified slightly below: // build.zig
const use_jemalloc = b.option(bool, "use_jemalloc", "test jemalloc") orelse false;
const exe_opts = b.addOptions();
exe_opts.addOption(bool, "use_jemalloc", use_jemalloc);
const exe_mod = b.createModule(.{
// ...
});
exe_mod.addOptions("build_opts", exe_opts);
// the jemalloc source code may not be downloaded, or maybe exe_mod wants to stay libc free
if (use_jemalloc) {
const jemalloc_mod = b.createModule(.{
// ...
});
exe_mod.addImport("use_jemalloc", jemalloc_mod);
} // main.zig
const functions = [_]Contructor{} ++
if (build_opts.use_jemalloc) @import("jemalloc").constructor else .{};
const build_opts = @import("build_opts");
// ... If I understand correctly, after this PR this code will no longer compile when |
In that case, I'd suggest just adding the module in the build script regardless of There could be other cases I've not thought of where this is more of a problem, though. If anyone has any, feel free to ask here, and we can see if this language change is problematic. |
It may be useful to label this PR as breaking, so more people are inclined to check it out. |
Allow specifying modules which the root module depends on. More complex graphs cannot currently be specified.
This commit makes some big changes to how we track state for Zig source files. In particular, it changes: * How `File` tracks its path on-disk * How AstGen discovers files * How file-level errors are tracked * How `builtin.zig` files and modules are created There is also one breaking change here, which is that `@import` of a non-existent module is now a compile error even if the import is not semantically analyzed. The original motivation here was to address incremental compilation bugs with the handling of files, such as ziglang#22696. To fix this, a few changes are necessary. Just like declarations may become unreferenced on an incremental update, meaning we suppress analysis errors associated with them, it is also possible for all imports of a file to be removed on an incremental update, in which case file-level errors for that file should be suppressed. As such, after AstGen, the compiler must traverse files (starting from analysis roots) and discover the set of "live files" for this update. Additionally, the compiler's previous handling of retryable file errors was not very good; the source location the error was reported as was based only on the first discovered import of that file. This source location also disappeared on future incremental updates. So, as a part of the file traversal above, we also need to figure out the source locations of imports which errors should be reported against. Another observation I made is that the "file exists in multiple modules" error was not implemented in a particularly good way (I get to say that because I wrote it!). It was subject to races, where the order in which different imports of a file were discovered affects both how errors are printed, and which module the file is arbitrarily assigned, with the latter in turn affecting which other files are considered for import. The thing I realised here is that while the AstGen worker pool is running, we cannot know for sure which module(s) a file is in; we could always discover an import later which changes the answer. So, here's how the AstGen workers have changed. We initially ensure that `zcu.import_table` contains the root files for all modules in this Zcu, even if we don't know any imports for them yet. Then, the AstGen workers do not need to be aware of modules. Instead, they simply ignore module imports, and only spin off more workers when they see a by-path import. During this phase, we work exclusively in absolute paths, to ensure consistency in terms of `NameTooLong` errors. After the AstGen workers all complete, we know that any file which might be imported is definitely in `import_table` and up-to-date. So, we perform a single-threaded graph traversal; similar to what `resolveReferences` plays for `AnalUnit`s, but for files instead. We figure out which files are alive, and which module each file is in. If a file turns out to be in multiple modules, we set a field on `Zcu` to indicate this error. If a file is in a different module to a prior update, we set a flag instructing `updateZirRefs` to invalidate all dependencies on the file. This traversal also discovers "import errors"; these are errors associated with a specific `@import`. There are two possible errors here: "module not found" when a module import uses an unmapped name, or "import outside of module root" when importing a file using `..` past the module root path. These errors have to be identified during this traversal because they depend on which module the file is in, which we don't know until now. For simplicity, `failed_files` now just maps to `?[]u8`, since the source location is always the whole file. In fact, this allows removing `LazySrcLoc.Offset.entire_file` completely, slightly simplifying some error reporting logic. File-level errors are now directly built in the `std.zig.ErrorBundle.Wip`. If the payload is not `null`, it is the message for a retryable error (i.e. an error loading the source file), and will be reported with a "file imported here" note pointing to the import site discovered during the single-threaded file traversal. The last piece of fallout here is how `Builtin` works. Rather than constructing "builtin" modules when creating `Package.Module`s, they are now constructed on-the-fly by `Zcu`. The map `Zcu.builtin_modules` maps from digests to `*Package.Module`s. These digests are abstract hashes of the `Builtin` value; i.e. all of the options which are placed into "builtin.zig". During the file traversal, we populate `builtin_modules` as needed, so that when we see this imports in Sema, we just grab the relevant entry from this map. This eliminates a bunch of awkward state tracking during construction of the module graph. It's also now clearer exactly what options the builtin module has, since previously it inherited some options arbitrarily from the first-created module with that "builtin" module! The actual effects of this commit are: * retryable file errors are now consistently reported against the whole file, with a note pointing to a live import of that file * incremental updates do not print retryable file errors differently between updates * incremental updates support files changing modules * incremental updates support files becoming unreferenced Resolves: ziglang#22696
This is a bit of a hack, but it only needs to stick around until ziglang/translate-c is ready to become the main `zig translate-c` implementation, at which point Aro will be removed from this repo.
See commit messages.
There is one breaking change here, which is that
@import("non_existent_module")
is now a compile error even if the import is never analyzed. This is going to fail CI at the moment, because Aro currently relies on the old behavior of accepting these imports.