AWS S3 Buckets and Google Cloud Storage

In this guide, we'll show you how to access and use external storage, such as S3 Buckets and Google Cloud Storage (GCS).

Step 1: access the bucket

To access external storage, press Cmd/Ctrl E (or click the “+” button below a block), and select the Bucket Access (S3/GCS) option from the Command Bar.

You need to select the Provider, and provide the Bucket Name. You can add a secret for private data access - typically, a cryptographic key created by the storage provider (see more details on how to use secrets here).

Step 2: mount the bucket

Once you've accessed the bucket, you can use it with any runtime. To do so, add a runtime in the language of your preference, open the runtime’s settings (the ⚙️ button next to the runtime's name on the sidebar), and go to the Mounts section to specify a mount point:

Save and restart, and you're done!

As an example, we have mounted here the nextjournal-maven bucket, and we use it from the Bash cell below:

ls -alF /nextjournal-maven

Configuring Google Cloud Storage

Configuring Bucket Access Control

GCS buckets access control model must be set to "fine-grained access control".

For existing buckets, change the access control in the Google Cloud Storage Dashboard by editing the according bucket, and choosing "Fine-grained":

For a new bucket, select "Fine-grained" when you're prompted to choose the access control:

Using Access Keys

You can obtain access to private buckets or write permissions with Google Cloud Storage S3 Interoperability Keys. For that, go to Google Cloud Storage Dashboard -> Settings -> Interoperability. If you have not set up interoperability before, first click Enable interoperability access. Then, either note the existing Access Key and Secret, or create a new key by clicking on "Create Key". Finally, add the key in Nextjournal as a secret in the form "<Access key>:<Secret>", without any whitespace.