CDO: Bulk Diagnosis of Model Output

Have you ever wanted a simple way to output global average values for each year from a series of monthly climate model output files? I have found myself in this situation several times, where I needed to check on various aspects of a climate model simulation (i.e. has the ocean temperature stabilized?). However, the data is usually distributed over many files, which makes this task difficult.

I’ve considered many ways to do this, such as:

  • creating a new NCL plotting script – but this takes too much time
  • running the NCAR AMWG Diagnostic Package – but this also takes time and produces a lot of plots that I don’t need
  • writing a script to scrape text out of the model log files (which actually works pretty well) – but these files only output certain values, which may not be what I’m looking for

It would be much more convenient to use CDO commands to get this job done efficiently and output it to the screen for a quick look. Thanks to the reply on this CDO forum post, now I know how to do this!

Suppose we have monthly output files like this:

CESM_00/atm/CESM_00.cam.h0.1850-01.nc
CESM_00/atm/CESM_00.cam.h0.1850-02.nc
CESM_00/atm/CESM_00.cam.h0.1850-03.nc
CESM_00/atm/CESM_00.cam.h0.1850-04.nc
CESM_00/atm/CESM_00.cam.h0.1850-05.nc
CESM_00/atm/CESM_00.cam.h0.1850-06.nc
CESM_00/atm/CESM_00.cam.h0.1850-07.nc
CESM_00/atm/CESM_00.cam.h0.1850-08.nc
CESM_00/atm/CESM_00.cam.h0.1850-09.nc
CESM_00/atm/CESM_00.cam.h0.1850-10.nc
CESM_00/atm/CESM_00.cam.h0.1850-11.nc
CESM_00/atm/CESM_00.cam.h0.1850-12.nc
...

My first thought was to try:

cdo  -yearmean  CESM_00/atm/*h0*

but part of the reason why this won’t work is because the CDO command wants an output file to put the result. But I really want it to just be written to the screen, so I don’t clutter up my data directories with random little unimportant files.

I also tried using this command:

cdo  -infon  -yearmean  -select,name=TS  CESM_00/atm/*h0*

  • -select,name=TS  :  this selects the variable(s) that we want to output
  • -yearmean  :  this uses the time coordinate variable to produce annual averages from any input, such as monthly, daily or sub-daily (you can also use -monmean)
  • -infon  :  this prints dataset information and lists the results by parameter name

And this actually works well for a single file, but with multiple files I get a huge mess of spatial averages of every variable, at every level, from every file.

The trick that was pointed out on the forum is that the wildcard file name  needs to be in quotes, like this:

> cdo -infon -yearmean -select,name=TS 'CESM_00/atm/*h0*'
cdo infon: Started child process "yearmean -select,name=TS CESM_00/atm/*h0* (pipe1.1)".
cdo(2) yearmean: Started child process "select,name=TS CESM_00/atm/*h0* (pipe2.1)".
cdo(3) select: 12% -1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name
 1 : 1850-07-01 00:00:00 0 55296 0 : 212.62 275.76 303.53 : TS 
 25% 2 : 1851-06-16 00:00:00 0 55296 0 : 213.73 275.63 302.70 : TS 
 37% 3 : 1852-06-16 00:00:00 0 55296 0 : 214.08 275.87 303.36 : TS 
 50% 4 : 1853-06-16 00:00:00 0 55296 0 : 214.64 275.76 302.89 : TS 
 62% 5 : 1854-06-16 00:00:00 0 55296 0 : 214.47 276.06 303.28 : TS 
 75% 6 : 1855-06-16 00:00:00 0 55296 0 : 214.45 276.06 303.55 : TS 
 87% 7 : 1856-06-16 00:00:00 0 55296 0 : 214.24 275.88 303.05 : TS 
 8 : 1857-06-16 00:00:00 0 55296 0 : 215.58 276.07 302.86 : TS 
 9 : 1858-01-01 00:00:00 0 55296 0 : 227.50 274.57 311.33 : TS 
cdo(3) select: Processed 5308416 values from 22560 variables over 96 timesteps ( 2.47s )
cdo(2) yearmean: Processed 5308416 values from 1 variable over 96 timesteps ( 2.47s )
cdo infon: Processed 497664 values from 1 variable over 9 timesteps ( 2.47s )

This output is a little messy, so we can clean it up using the -s flag to “silence” some of that extra information:

> cdo -s -infon -fldmean -yearmean -select,name=TS 'CESM_00/atm/*h0*'
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name
 1 : 1850-07-01 00:00:00 0 1 0 : 286.35 : TS 
 2 : 1851-06-16 00:00:00 0 1 0 : 286.21 : TS 
 3 : 1852-06-16 00:00:00 0 1 0 : 286.23 : TS 
 4 : 1853-06-16 00:00:00 0 1 0 : 286.22 : TS 
 5 : 1854-06-16 00:00:00 0 1 0 : 286.40 : TS 
 6 : 1855-06-16 00:00:00 0 1 0 : 286.40 : TS 
 7 : 1856-06-16 00:00:00 0 1 0 : 286.25 : TS 
 8 : 1857-06-16 00:00:00 0 1 0 : 286.32 : TS

Of course then you can make this into a handy little alias to your ~/.bashrc or ~/.profile file, so you don’t have to type as much like this:

cdo_avgTS   'CESM_00/atm/*h0*'

 

Leave a Reply

Your email address will not be published. Required fields are marked *