[julia] Install the CompilerExplorer.jl after installing the compiler

[giordano]

This is my initial attempt to address #1259, but there are still things I don't understand, so opening as draft.

CC: @vchuravy.

[giordano]

Can I define custom fields in this yaml file? I did it for convenience, to avoid repeating some things, but if it doesn't work I'll change this;

[mattgodbolt]

you can; and that's the intention to let you cascading-ly refer to things later on as you do.

[giordano]

I have no clue what path destination is, I saw it in

infra/bin/yaml/haskell.yaml

Line 14 in f7d9fb5

- ./configure --prefix={destination}/{dir}

but I don't know if {destination}/depot is something like /opt/compiler-explorer/julia-{name} or what.

[mattgodbolt]

You may need to quote some of these things to make them valid yaml

[giordano]

Is bin/julia (== {julia_exe}) a sensible way to call the julia executable after it has been installed? I followed the check_exe example.

[vchuravy]

Suggested change

	- JULIA_DEPOT_PATH={depot} {julia_exe} -e 'using Pkg; Pkg.add(; url="https://github.com/giordano/CompilerExplorer.jl", rev="{ce_revision}"); Pkg.precompile()'
	- JULIA_DEPOT_PATH={depot}: {julia_exe} -e 'using Pkg; Pkg.add(; url="https://github.com/giordano/CompilerExplorer.jl", rev="{ce_revision}"); Pkg.precompile()'

Any reason for using {depot} instead of {destination}/local/share/julia?

We likely also need a project + manifest in the LOAD_PATH

[giordano]

I was planning adding the package to the global environment, so that we don't have to deal with adding a Project.toml/Manifest.toml here (the latter file being problematic, since it's different for each julia version). But I'm open to better solutions.

[vchuravy]

So if you use the local depot https://docs.julialang.org/en/v1/base/constants/#Base.DEPOT_PATH

Then we don't need to modify the depot path later one.

I don't think we want to use the default project since the user might try to modify it and this is read-only (might also be fine).

You need : after the path so that we still have the system depot available for the stdlib

[giordano]

In order to set JULIA_DEPOT_PATH=/path/to/depot, in https://github.com/compiler-explorer/compiler-explorer/blob/042e2b1c02416ac7039d050bbccba726e1e7ddaf/lib/compilers/julia.ts we need to override getDefaultExecOptions to add JULIA_DEPOT_PATH=... to the environment, right?

[partouf]

In order to set JULIA_DEPOT_PATH=/path/to/depot, in https://github.com/compiler-explorer/compiler-explorer/blob/042e2b1c02416ac7039d050bbccba726e1e7ddaf/lib/compilers/julia.ts we need to override getDefaultExecOptions to add JULIA_DEPOT_PATH=... to the environment, right?

correct

[mattgodbolt]

Hi @giordano any desire to complete this PR, or shall I close it? Thanks!

[giordano]

I'd like to finish the PR, but I need to understand better how caching works in your infrastructure.

As it is now, the Julia driver script is self-sufficient and doesn't need anything extra, which is nice and simple. However turning it into a proper package would let us (a) cache native code to speed up the workload a little bit, and (b) more easily test the entire workflow.

The point with the package approach is that we want to install the package, have it precompiled, and the cached file(s) be automatically reused every time. Is that doable? Where we should do this installation? In Julia there's the concept of a depot where this code is looked for, but its location can be controlled by setting an environment variable (which is what I was trying to do above).