// REANALYSIS ENSEMBLE SERVICE (RES)

MERRA ANALYTICS INTERFACE

The Reanalysis Ensemble Service (RES) allows users to perform queries on reanalysis datasets from MERRA data. RES was created to address the Big Data challenges of climate science. As the availability and volume of Earth data grow, researchers spend more time downloading and processing their data than doing science. The NASA Center for Climate Simulation (NCCS) has developed the Reanalysis Ensemble Service (RES), a high-performance big data analytics framework built on Apache Hadoop, to allow researchers to leverage our compute power to analyze large datasets located at the NCCS through a web-based interface, thereby eliminating the need to download the data. 

RES provides access to a suite of “canonical operations”—min, max, sum, difference, average, root mean square, anomaly, and standard deviation— that researchers can combine to develop various workflows. RES uses MapReduce to efficiently processing huge datasets within limited memory spaces at interactive and batch response times. These operations and datasets can be accessed via RES using applications written by the user.

The RES interface supports web service access to consumer applications, a graphical user interface for interactive requests, a command line interface for users familiar with basic RES commands, and advanced programmatic access for Python-savvy users. RES allows users to compute close to the data without downloading input datasets.

// ARCHITECTURAL COMPONENTS

There are three components to the RES architecture:

  1. Client run software, either Jupyter Notebook or Python scripts, that is installed on the user's system via Python.
  2. Apache HTTP web server that listens for external requests, parses them, and submits them to the analytics server.
  3. Backend analytics code.  This utilizes Cloudera/Hadoop to partition the work to the available worker nodes.

The user invokes the client to access the Apache HTTP web server which forwards the request to the backend server. Results are returned to the user's system as NetCDF files.


// USING RES

Everything you need to know about how to work with RES in one place.

  • Installation
  • How to use
  • Operations
  • Downloading results
  • Example Code
LEARN MORE

RES DATA COLLECTIONS

See the Earth data holdings available through RES.


DATASETS

Service Consultation

Our goal is to support your unique and evolving high performance computing requirements. The more we know about your computational challenges, the better we can innovate to provide solutions.

For support, e-mail us at support@nccs.nasa.gov with subject line: RES.

CONTACT US