Simon Danisch / Jul 25 2019

PackageCompiler

Features

  • create a system image containing precompile statements
  • convenience API for the above: compile_incremental(packages...)
  • create a shared library out of a Julia script
  • create an executable
  • https://nextjournal.com/julia

History

State

  • written very hastily
  • my main use case is satisfied
  • infamous sd-notomls
  • no fun to work on in my free-time

How to rewrite PackageCompiler

At its Core:

Code you want to get into your system Image:

Base.__init__(); Sys.__init__()
function test() # This will be added to your image
  println("hi binaries")
end
test() # Global scope won't stay
precompile.jl
Julia

Compile it:

lib = abspath(joinpath(
  Sys.BINDIR, Base.PRIVATE_LIBDIR
))
img = unsafe_string(Base.JLOptions().image_file)
# compile image
run(`julia -J$(img) -O3 --output-o sys.a /precompile.jl`)
# Link it
run(`g++ -m64 -shared -fPIC -L$(lib) -L$(dirname(lib)) -o sys.so -Wl,--whole-archive sys.a -Wl,--no-whole-archive`);
154.9s
Julia
Julia 1.1
run(`julia -J/sys.so -e 'test()'`);
0.8s
Julia
Julia 1.1

Snooping

run(`julia --compile=all -O0 -g0 --trace-compile=/results/snoop.jl /precompile.jl`);
0.9s
Julia
Julia 1.1
snoop.jl
println(read(
snoop.jl
, String))
0.6s
Julia
Julia 1.1

Problems

  • reverse engineer this
  • what to snoop?
  • package dependencies
  • projects & system images

Code behaves differently with output-o

]add PackageCompiler
65.9s
Julia
Julia 1.1
using PackageCompiler
println(PackageCompiler.PrecompileCommand(
snoop.jl
))
0.5s
Julia
Julia 1.1

Final Remarks

  • if anyone wants to rewrite PackageCompiler: read through our hacks and ask :P
  • Try out David Anthoff Visual Studio plugin