Skip to content
All regions operational
NorthmereData Commons
Home/Documentation

Documentation

Everything you need to find a collection, move it to your own storage, read it with common tools and cite it correctly.

Start here

Northmere hosts three kinds of scientific data, and nearly all of it is large. The documentation is organised around the four things people do with it: find, fetch, read and cite.

Find
Use the catalogue for the largest collections, or the STAC interface to search the full index of 2,184 collections by area, time range and variable.
Fetch
The bulk download guide covers aria2 manifests, curl, rclone and the Globus-style transfer endpoint, with advice on connection counts.
Read
The file format notes list the formats in use (Zarr, NetCDF-4, CRAM, Parquet, cloud-optimised GeoTIFF) and the libraries that open them.
Cite
The citation guide shows the exact reference to use for each collection version.

A first download in five minutes

  1. Pick a small collection, for example SAT-2074 (5.8 TB) or CLM-1177 (3.4 TB), and open its page.
  2. Download the INDEX.txt in the first directory. It is a few kilobytes and needs no account.
  3. Request a Researcher account if you want the full signed manifest. Verification takes under one working day.
  4. Run the aria2 command from the collection page and leave it running. Interrupted transfers resume where they stopped.
  5. Verify with sha256sum -c SHA256SUMS inside the target directory.

Before you start a multi-terabyte transfer, check your free disk space and the monthly allowance of your tier. A 40 TB collection will not fit in a 12 TB Researcher allowance, but it can be fetched in slices: every collection is split into directories of 1 to 8 TB that you can request separately.

Service limits at a glance

TierConnectionsThroughputMonthly egressSigned URL lifetime
Open4400 Mbit/s500 GB per IPnone
Researcher162 Gbit/s12 TB per user7 days
Associate325 Gbit/s300 TB pooled14 days
Full6410 Gbit/s2 PB pooled30 days

Getting help

Support requests go through the contact page. Please include the collection code, the exact command and the first error line. Operational problems are posted on the status page before they are answered individually.