Importing and Exporting

Openness is at the core of Nextjournal's design philosophy. All work produced on Nextjournal is easily shared, remixed, and indexed by search engines. Nextjournal offers import and privacy options for works in progress and export tools to give you total control of your research.

1.
Import

1.1.
Data, Images, and Files

Inserting or adding new content to Nextjournal

Upload data and images to Nextjournal using the File and Image commands in the ➕ insert menu. For example, File → matrix-data.csv renders:

matrix-data.csv

Both the ➕ insert menu and the ··· action menu are exposed when selecting or hovering over notebook elements like paragraphs or code cells.

Files and images are saved in content addressable storage. This means that the information is immutable and any changes creates an entirely new file. This is one way Nextjournal guarantees the integrity of your data.

1.2.
Jupyter, Markdown, and R Markdown

Have existing work in Jupyter Notebook, Markdown, or R Markdown? Import it to Nextjournal to take advantage of automatic, full-stack versioning, simple cloud storage, powerful collaboration tools, elegant dependency management, and transparent containerization technology.

Supported import formats include Jupyter Notebooks (.ipynb), R Markdown (.rmd), Markdown (.md), and JATS XML, in addition to the raw internal .njson representation.

To import, find the option above the language templates on the new article page. You can upload your work from your local machine or enter a URL to grab a raw file from the web.

The import option in Nextjournal

1.2.1.
Jupyter

Nextjournal can import Jupyter notebooks version 3 and above. The import function will render the entire notebook, including code cell output.

The imported notebook will default to a Jupyter kernel runtime but immediately benefit from many of Nextjournal's most powerful features like version control and collaboration.

1.2.2.
Markdown

Nextjournal supports ATX style headers.

When importing, header order and consistency will be enforced. If the document starts with a single # header , it will be used as the article title. Otherwise, Untitled will be used.

1.2.2.1.
A Single Header
# Lorem Ipsum

Lorem ipsum dolor sit amet.

Sed ut perspiciatis unde omnis.
Markdown

⤷ renders:

Result of a single header

1.2.2.2.
A Structured Document
# Section 1
## 1.1
## 1.2
# Section 2
## 2.1
## 2.2
Markdown

⤷ renders:

A structured document

1.2.2.3.
Code, Text, and Media Formatting

Fenced code blocks using three backticks ``` are imported as Nextjournal code nodes if they are using one of the supported languages or as code-listing for other languages.

Inline Code

This is `inline code`. 
Markdown

⤷ renders:

This is inline code.

Fenced Code Block

```clojure(prn "Gets imported")```
Markdown

⤷ renders:

clojure(prn "Gets imported")

Supported Language Runtime

```r
print("Lorem ipsum")
```
Markdown

⤷ renders:

print("Lorem ipsum")

Note that this is an executable R runtime cell.

Supported Language Runtime Without Execution

```python no-exec
print "Gets imported as a non executable code-listing"
```
Markdown

⤷ renders:

print "Gets imported as a non executable code-listing"
Python

Note that this is a Code Listing and cannot be executed.

Basic formatting tags, em, strong, hr, and br, are supported.

Images and image references are downloaded and stored in Nextjournal.

1.3.
Docker Images

Nextjournal allows users to mount and run any Docker image. For example, this is the default Nextjournal operating system:

head -n 2 /etc/os-release

Using the Import Docker Environment command in the ➕ insert menu, it is easy to add any image from the web. For example, first load the Bash official image on Docker Hub:

bash
Download as Docker image from:
This image was imported from: bash:latest

Then create a new cell, change the environment to the new Docker container (in this case is is called bash), and inspect the operating system:

head -n 2 /etc/os-release

Nextjournal will store this particular image, meaning that upstream changes will not affect your notebook.

1.4.
External Repositories

Nextjournal supports both public and private repositories on GitHub, Amazon S3, and Google Cloud Storage.

Authentication tokens for private repositories are not stored in the notebook. They are associated with the author's profile and can be created by clicking on your user avatar and selecting Settings → Secrets.

Notebooks that feature private repositories can be remixed and shared like any other notebook. However, tokens to use private repositories will need to be supplied separately to any user who wants to run the notebook cells that depend on this data.

Data from a private repository is only held in memory for as long as the associated runner is active. Nextjournal does not store private data on our servers and all secrets are kept secure using Vault.

1.4.1.
GitHub

Select Import GitHub Repository from the ➕ insert menu.

  • Repository Name: The organization and repository name. For example, the URL https://github.com/schmudde/jupyter-git would yield schmudde/jupyter-git.
  • Ref: the field can be empty for the master branch or given a specific branch name or commit SHA.

⤷ renders:

Mount the repository by selecting + Add mount under the runtime settings. All possible mount points are available under Source. The Destination is given a default but can be customized.

Add a mount

Now it is easy to read the mounted GitHub repository's README.md:

head /jupyter-git/README.md

1.4.2.
Amazon S3 and Google Cloud Storage

Select Bucket Access (S3 / GCS) from the ➕ insert menu.

⤷ renders:

nextjournal-s3-demo
Public

Mount the repository by selecting + Add mount under the runtime settings. All possible mount points are available under Source. The Destination is given a default but can be customized.

Add a mount

View the contents of the bucket.

ls -lah /nextjournal-s3-demo

Working with remote data is just like working with any other information on Nextjournal. For example, Nextjournal will automatically render this bucket's thumbs-up-2.png if copied to results:

cp /nextjournal-s3-demo/thumbs-up-2.png /results/thumbs-up-2.png

Refer to Understanding Results for more information on how results are displayed on Nextjournal.

2.
Export

2.1.
Markdown

Nextjournal exports a .njmd markdown document that will render your notebook, including code cells, output, and media, in any markdown rendering software. The file also features runtime metadata that makes importing back into Nextjournal seamless.

Some advanced Nextjournal features are not exportable, including references in code nodes, code includes, and transclusions. Nodes and references that require information that cannot be easily expressed in markdown are exported as a markdown reference [name][target]. The target contains an id that is used to lookup extra information in the metadata code block.

2.2.
Data

The results of processing data uploaded or generated in Nextjournal is easily exported.

The example below takes the data uploaded in section 1.1,  matrix-data.csv↩, doubles the values of every number in the matrix, and makes it available for downloading in a file called output.txt.

# Read the .csv uploaded in section 1.1
matrix_data <- read.csv(file=matrix-data.csv↩, header=F, sep=" ")

# Create a list of two matrix elements
.list <- list(as.matrix(matrix_data), as.matrix(matrix_data)) 

# Reduce the elements in the list and write to output
write(Reduce('+', .list), "/results/output.txt")
output.txt

Click the cloud button to the right of the result to download.

2.3.
Docker Images

Because Nextjournal environments are modular components independent of the article, it is easy to download your dependency stack and use it in a local context. Select Export the environment under the runtime settings to make the Docker image available in the Google Container Registry. The image will be downloadable from the image registry in the Google Cloud Platform.

Export an environment and copy the URL

Copy the image's unique url and download it with a simple docker pull command on your local machine. The URL is located just below Export the Environment.

For example, this command will pull this notebook's environment:

docker pull eu.gcr.io/nextjournal-com/environment@sha256:f461e4e5aa068c5bd2eea834d1bac9ab280efd37c796f72dd9f980090aaf4432
Bash
© 2018 Nextjournal GmbH