Working With Code and Data
Nextjournal currently supports running the following programming languages: Python, R, Julia, Clojure and Bash. From time to time, you may stumble upon notebooks that have other (experimental) languages enabled but we won’t cover those here as they are not officially supported yet.
All programming in Nextjournal starts by adding a code cell. Click the + button between nodes and select Code Cell: [Language] for the programming language you want to work with:
When you insert your first code cell, you’ll see that Nextjournal automatically inserts a matching runtime. Runtimes are shown in the sidebar to the left and in a dedicated Runtimes section at the bottom of your notebook. You can find out about the details of how runtimes work in the Runtimes & Environments guide.
Once you have typed some code into a code cell, there are various ways to run it:
Running a single cell: Click the play button in the lower right corner of a code cell (or by press
Run cell and focus next: Pressing
Shift+Enter will run the current cell you are in and jumps to the next cell using the same language runtime. If no next cell exists, Nextjournal will insert a new cell instead.
Running all cells: This option (and more) is available from the run menu in the top bar of the Nextjournal editor:
When you run the cell, you will see that its status (in the cell’s lower left corner) changes. Also depending on the cell’s runtime, the status might be something like Waiting or Executing or, once the run is complete, Done.
During a cell’s run, the associated runtime’s status (look at the runtime in the sidebar) also changes. The first run might take a little longer because it will need to boot up the runtime first. Once the runtime is running, code execution should be fairly quick (depending on what you are trying to accomplish).
Code Completion & Docs
Whenever you select some code in a code cell, you will see that a selection toolbox pops up that presents you with a Complete and Docs option.
If you click Complete (or press
Tab), Nextjournal will try to autocomplete the currently selected piece of code:
Similarly, if you click Docs (or press
Shift+Tab), Nextjournal will try to present you with documentation about the currently selected piece of code:
Please mind that for both of these features to work, your code cell’s associated runtime has to be running. Runtimes shut down automatically after 20 minutes of idle time so you might need to run a cell first to boot its runtime up.
If a code cell writes output to
stdout, e.g. when installing packages or listing a directory, it will display it in a scrolling section right below your code:
If you produce an error while running your program, Nextjournal will try to match the error to an offending line. This will work more or less well depending on the type of error that occurs. You will find an Ask for help button right next to the offending line. You can use it to give our staff permission to help out with your notebook if you happen to be stuck:
In Nextjournal, you have full access to your runtime environment’s file system, so there are no limitations on where you can save your results to. However, bear in mind that all results you write to a runtime’s environment’s file system are transient. That means once the runtime shuts down or you reset it, all results are lost and all cells will have to be re-run to generate their results.
To prevent this, Nextjournal has a special directory called
/results. Whenever a code cell writes a file to
/results, Nextjournal will persist it and version it together with the notebook’s content. This means that whenever you’re going back in time using the Notebook History feature, any previous persisted results will be restored along with the notebook’s previous content.
Furthermore, any result written to
/results will display it below your code using a viewer that’s automatically selected based on the result file’s mime type:
The simplest representation for this is a file but based on the mime type, Nextjournal might display it as text, data structure, table, image, or interactive plotly graph. You can also change the pre-selected viewer (given that your mime type has more than one corresponding view option):
Please mind that you won’t be able to list the contents of
/results. Its sole purpose is to write to it so that Nextjournal can version and display it.
Working With Data
There are various ways to get data into a Nextjournal notebook (e.g. via GitHub or Amazon S3 or Google Cloud) but for this guide we will focus on working with uploaded and downloaded files.
Working With Uploaded Files
Everything you upload to your notebook will be automatically versioned together with all other content and code. To upload a file up to 500 MB, simply drag it into this notebook or click the + button between nodes and select File:
Once your file is uploaded, you can reference it in any code cell by selecting some text and selecting + from the selection toolbox or by pressing
Large Files & Versioned Downloads to /results
To store files larger than 500 MB in a notebook, or simply to acquire them from elsewhere on the internet (websites, repositories, cloud storage like Dropbox), you can use the
wget programs in a Bash cell, and download or copy to the special
/results directory. Below, these programs are used to download two formats of the CIFAR-10 machine learning dataset:
Files placed in
/results are moved into the same versioned storage system as uploaded files. As above, after downloading you can lock the cell using the ··· menu. This ensures the files will remain untouched, so you never have to wait for them to download again, and also save the file host's bandwidth.
Referencing Files Produced by Other Code Cells
You can use the same interaction as for uploaded files (select code then + or press
Ctrl/Cmd+E) to reference any file that was written to
/results no matter which code cell produced it. The file's name will be preserved for programs that need it.
For use cases where files must be arranged in a specific directory structure, the most efficient method is to create symlinks within the directory structure, pointing to the files in storage. As an example, this would allow the Keras neural network library to find the CIFAR-10 data downloaded above:
However, in the event that read-write access to a file is required (a notable case is many HDF5 files),
cp in a Bash cell or any in-language file copying function can be used to copy files from storage into the local filesystem.
Any modified data files can then be copied into
/results to save and version them.
That’s it for the Working with code and data guide. We recommend you play around with what you learned here for a bit and if you feel you confident or just want to learn more, head over to the Runtimes & Environments guide to learn more about the underpinnings behind code execution in Nextjournal.