DKRZ Tech Talks

Intake - Taking the pain out of data access

by Aaron Spring (MPIM)

Europe/Berlin
Description
Recording on Youtube (external site)
Slides at dkrz gitlab.
 
Intake lets users easily load datasets in Python. In this Tech Talk, Aaron Spring from the Max Planck Institute for Meteorology will show how to use intake on the mistral supercomputer for loading locally available and remote data.
 
Most climate science workflows start with data access. Especially when working with observations, handling and aligning of metadata is a tedious and repetetive exercise.
In the scientific python ecosystem, intake lets you access datasets from pre-defined catalogs and loads them with xarray and dask. The data handling for a multi-model CMIP6 comparison analysis takes only 5 minutes with intake.

Contents:
  • intake-esm on mistral
  • intake-xarray for ICDC data: Catalog for Ocean data provided by Remon Sadikni
  • intake-xarray for remote sources