TS
Tom Short / Dec 12 2018

Executing Julia in the Browser


1.
Compiling Julia to Webassembly

Sadly, compiling Julia to webassembly is not fully working yet, but at least it compiles the core dependencies to boot Julia and starts the compiler. From here on, we just need to figure out a better way to map the encountered errors to Julia source and fix one issue after the other.

1.1.
Setup & Compilation

Add some dependencies we need:

apt-get update
apt-get install git python -y
# Julia dependencies
# https://github.com/JuliaLang/julia/#required-build-tools-and-external-libraries
sudo apt-get install build-essential libatomic1 python gfortran perl wget m4 cmake pkg-config
56.3s
Language:Bash

Install Webassembly SDK

git clone https://github.com/juj/emsdk.git
cd emsdk
./emsdk install latest
./emsdk activate latest
36.7s
Language:Bash

Build Julia master

git clone https://github.com/JuliaLang/julia.git
cd julia
make -j8
3479.2s
Language:Bash


For now, we're just partially build julia with emcc to webassembly, just to get a first REPL running.

cd julia
git checkout kf/wasm
# add a Make.user
echo "override CC=emcc
override CXX=emcc
JULIACODEGEN=none
CFLAGS=--source-map-base http://localhost:8888/ -s WASM=1
override OS=wasm
override JULIA_THREADS=0
override USE_SYSTEM_BLAS=1
override USE_SYSTEM_LAPACK=1
override USE_SYSTEM_LIBM=1
override USE_SYSTEM_DSFMT=1
override DISABLE_LIBUNWIND=1" >> Make.user
# Make a backup of the flisp files
# Since we shouldn't 
cp src/julia_flisp.boot src/julia_flisp.boot.bak
cp src/julia_flisp.boot.inc src/julia_flisp.boot.ink.bak
1.1s
Language:Bash

Compile C dependency

cd julia
make -C deps/ clean-utf8proc
make -C deps/ install-utf8proc
2.7s
Language:Bash
touch julia/src/julia_flisp.boot
touch julia/src/julia_flisp.boot.inc
0.1s
Language:Bash

1.1.1.
Start a Julia Webserver for the shared filesystem

bash can't actually reach the outside of the container, so we need to run the webserver in Julia! We also need to start it before compiling, because we need to get the URL that it gets proxied to!
pkg"add HTTP#master"
ispath("/shared/julia") || mkdir("/shared/julia")
cd("/shared/julia")
using HTTP
function serve_file(req::HTTP.Request)
  for root in ("", "/src", "/src/flisp", "/src/support")
    file = abspath(pwd() * root * req.target)
    isfile(file) && return HTTP.Response(200, [], body = read(file))
  end
  return HTTP.Response(404)
end
handler = HTTP.RequestHandlerFunction(serve_file)
@async HTTP.Servers.listen("0.0.0.0", 9998) do http
    HTTP.handle(handler, http)
end
base_url = strip(ENV["NEXTJOURNAL_RUNTIME_SERVICE_URL"]) # the proxy url
url = base_url * "/hello.html"
base_url*"/"
37.1s
Language:Julia

Now we can use emcc to link the program and generate javascript + html to load our webassembly binary. Now that we have the server running, we can also reference the proxy url for the source map:

cd julia
touch src/julia_flisp.boot
touch src/julia_flisp.boot.inc
make -C src
emcc -Isrc/support -Lusr/lib -ljulia ui/repl-wasm.c --preload-file base/boot.jl --source-map-base nil↩ -g4 -s -s WASM=1 -s ASSERTIONS=1 -s ALLOW_MEMORY_GROWTH=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0 -o hello.html
12.8s
Language:Bash

Copy the new files to the file system shared with the Julia runner:

cp -ru /julia/* /shared/julia/.
32.4s
Language:Bash

Display the html we generate & host:

HTML("""
<iframe id="wasm" src=$(repr(url)) style="width:100%; height=800" frameborder="0">
</iframe>
""")
1.4s
Language:Julia

At this point, the WebAssembly code has run in the browser, but it has failed. It has failed during the initialization phase here where jl_init reads in the "boot.jl" file. That's the point where Julia needs code generation and LLVM.

1.2.
What's Next?

There's still a bit of work ahead before Julia or Julia code can run with WebAssembly, but this code helps. One option to proceed is to improve Julia's interpreter well enough to run without code generation (with JULIACODEGEN=none). This doesn't currently work (issue), but this approach could provide a lightweight runtime.

Compiling LLVM and code generation to WebAssembly is another route to running Julia in the browser. An older version of LLVM has been compiled with Emscripten here. In general, compiling libraries from C and other languages will help support more Julia features (especially BLAS). To that end, targets for BinaryBuilder for wasm32-unknown-unknown and wasm32-unknown-emscripten have been started. These may make it easier to port libraries (including LLVM) to WebAssembly.

For static compilation of Julia code to WebAssembly, the changes in the kf/wasm branch have produced a nearly working version of libjulia. That could be integrated into Charlotte or one of the other approaches to generating static WebAssembly code. That would allow rich support of arrays, strings, and dictionaries.