Skip to content

Add julia pass pipeline to opt #46191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Sep 16, 2022
Merged

Add julia pass pipeline to opt #46191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3d573bf
Use NewPM for optimization unless ASAN is in effect
pchintalapudi Jul 26, 2022
a2be2d4
Disable NewPM by default
pchintalapudi Aug 11, 2022
4464bf5
Add julia pass pipeline to opt
pchintalapudi Jul 27, 2022
285c4da
Update llvm devdocs
pchintalapudi Jul 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 37 additions & 13 deletions doc/src/devdocs/llvm.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,36 @@ Julia dynamically links against LLVM by default. Build with `USE_LLVM_SHLIB=0` t

The code for lowering Julia AST to LLVM IR or interpreting it directly is in directory `src/`.

| File | Description |
|:------------------- |:---------------------------------------------------------- |
| `builtins.c` | Builtin functions |
| `ccall.cpp` | Lowering [`ccall`](@ref) |
| `cgutils.cpp` | Lowering utilities, notably for array and tuple accesses |
| `codegen.cpp` | Top-level of code generation, pass list, lowering builtins |
| `debuginfo.cpp` | Tracks debug information for JIT code |
| `disasm.cpp` | Handles native object file and JIT code diassembly |
| `gf.c` | Generic functions |
| `intrinsics.cpp` | Lowering intrinsics |
| `llvm-simdloop.cpp` | Custom LLVM pass for [`@simd`](@ref) |
| `sys.c` | I/O and operating system utility functions |
| File | Description |
|:-------------------------------- |:------------------------------------------------------------------ |
| `aotcompile.cpp` | Legacy pass manager pipeline, compiler C-interface entry |
| `builtins.c` | Builtin functions |
| `ccall.cpp` | Lowering [`ccall`](@ref) |
| `cgutils.cpp` | Lowering utilities, notably for array and tuple accesses |
| `codegen.cpp` | Top-level of code generation, pass list, lowering builtins |
| `debuginfo.cpp` | Tracks debug information for JIT code |
| `disasm.cpp` | Handles native object file and JIT code diassembly |
| `gf.c` | Generic functions |
| `intrinsics.cpp` | Lowering intrinsics |
| `jitlayers.cpp` | JIT-specific code, ORC compilation layers/utilities |
| `llvm-alloc-helpers.cpp` | Julia-specific escape analysis |
| `llvm-alloc-opt.cpp` | Custom LLVM pass to demote heap allocations to the stack |
| `llvm-cpufeatures.cpp` | Custom LLVM pass to lower CPU-based functions (e.g. haveFMA) |
| `llvm-demote-float16.cpp` | Custom LLVM pass to lower 16b float ops to 32b float ops |
| `llvm-final-gc-lowering.cpp` | Custom LLVM pass to lower GC calls to their final form |
| `llvm-gc-invariant-verifier.cpp` | Custom LLVM pass to verify Julia GC invariants |
| `llvm-julia-licm.cpp` | Custom LLVM pass to hoist/sink Julia-specific intrinsics |
| `llvm-late-gc-lowering.cpp` | Custom LLVM pass to root GC-tracked values |
| `llvm-lower-handlers.cpp` | Custom LLVM pass to lower try-catch blocks |
| `llvm-muladd.cpp` | Custom LLVM pass for fast-match FMA |
| `llvm-multiversioning.cpp` | Custom LLVM pass to generate sysimg code on multiple architectures |
| `llvm-propagate-addrspaces.cpp` | Custom LLVM pass to canonicalize addrspaces |
| `llvm-ptls.cpp` | Custom LLVM pass to lower TLS operations |
| `llvm-remove-addrspaces.cpp` | Custom LLVM pass to remove Julia addrspaces |
| `llvm-remove-ni.cpp` | Custom LLVM pass to remove Julia non-integral addrspaces |
| `llvm-simdloop.cpp` | Custom LLVM pass for [`@simd`](@ref) |
| `pipeline.cpp` | New pass manager pipeline, pass pipeline parsing |
| `sys.c` | I/O and operating system utility functions |

Some of the `.cpp` files form a group that compile to a single object.

Expand Down Expand Up @@ -77,12 +95,18 @@ LLVM tools as usual. `libjulia` can function as an LLVM pass plugin and can be
loaded into LLVM tools, to make julia-specific passes available in this
environment. In addition, it exposes the `-julia` meta-pass, which runs the
entire Julia pass-pipeline over the IR. As an example, to generate a system
image, one could do:
image with the old pass manager, one could do:
```
opt -enable-new-pm=0 -load libjulia-codegen.so -julia -o opt.bc unopt.bc
llc -o sys.o opt.bc
cc -shared -o sys.so sys.o
```
To generate a system image with the new pass manager, one could do:
```
opt -load-pass-plugin=libjulia-codegen.so --passes='julia' -o opt.bc unopt.bc
llc -o sys.o opt.bc
cc -shared -o sys.so sys.o
```
This system image can then be loaded by `julia` as usual.

It is also possible to dump an LLVM IR module for just one Julia function,
Expand Down
41 changes: 1 addition & 40 deletions src/aotcompile.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -562,7 +562,7 @@ void jl_dump_native_impl(void *native_code,
addOptimizationPasses(&optimizer, jl_options.opt_level, true, true);
addMachinePasses(&optimizer, jl_options.opt_level);
#else
NewPM optimizer{std::move(TM), getOptLevel(jl_options.opt_level), {true, true, false}};
NewPM optimizer{std::move(TM), getOptLevel(jl_options.opt_level), OptimizationOptions::defaults(true, true)};
#endif

Type *T_size;
Expand Down Expand Up @@ -961,45 +961,6 @@ void jl_add_optimization_passes_impl(LLVMPassManagerRef PM, int opt_level, int l
addOptimizationPasses(unwrap(PM), opt_level, lower_intrinsics);
}

// new pass manager plugin

// NOTE: Instead of exporting all the constructors in passes.h we could
// forward the callbacks to the respective passes. LLVM seems to prefer this,
// and when we add the full pass builder having them directly will be helpful.
static void registerCallbacks(PassBuilder &PB) {
PB.registerPipelineParsingCallback(
[](StringRef Name, FunctionPassManager &PM,
ArrayRef<PassBuilder::PipelineElement> InnerPipeline) {
#define FUNCTION_PASS(NAME, CREATE_PASS) if (Name == NAME) { PM.addPass(CREATE_PASS); return true; }
#include "llvm-julia-passes.inc"
#undef FUNCTION_PASS
return false;
});

PB.registerPipelineParsingCallback(
[](StringRef Name, ModulePassManager &PM,
ArrayRef<PassBuilder::PipelineElement> InnerPipeline) {
#define MODULE_PASS(NAME, CREATE_PASS) if (Name == NAME) { PM.addPass(CREATE_PASS); return true; }
#include "llvm-julia-passes.inc"
#undef MODULE_PASS
return false;
});

PB.registerPipelineParsingCallback(
[](StringRef Name, LoopPassManager &PM,
ArrayRef<PassBuilder::PipelineElement> InnerPipeline) {
#define LOOP_PASS(NAME, CREATE_PASS) if (Name == NAME) { PM.addPass(CREATE_PASS); return true; }
#include "llvm-julia-passes.inc"
#undef LOOP_PASS
return false;
});
}

extern "C" JL_DLLEXPORT ::llvm::PassPluginLibraryInfo
llvmGetPassPluginInfo() {
return {LLVM_PLUGIN_API_VERSION, "Julia", "1", registerCallbacks};
}

// --- native code info, and dump function to IR and ASM ---
// Get pointer to llvm::Function instance, compiling if necessary
// for use in reflection from Julia.
Expand Down
11 changes: 8 additions & 3 deletions src/jitlayers.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,14 @@ struct OptimizationOptions {
bool lower_intrinsics;
bool dump_native;
bool external_use;

static constexpr OptimizationOptions defaults() {
return {true, false, false};
bool llvm_only;

static constexpr OptimizationOptions defaults(
bool lower_intrinsics=true,
bool dump_native=false,
bool external_use=false,
bool llvm_only=false) {
return {lower_intrinsics, dump_native, external_use, llvm_only};
}
};

Expand Down
Loading