Let’s say you have a bunch of daily NetCDF files for several years, each with many variables, and you want to extract a single variable and concatenate it into a single output file in one step. I recently had this scenario dealing with model output from the NCAR Community Earth System Model (CESM).
AQUA_ZM_00_1.9x2.5_344.cam.h1.0000-01-01-00000.nc AQUA_ZM_00_1.9x2.5_344.cam.h1.0000-01-02-00000.nc AQUA_ZM_00_1.9x2.5_344.cam.h1.0000-01-03-00000.nc AQUA_ZM_00_1.9x2.5_344.cam.h1.0000-01-04-00000.nc AQUA_ZM_00_1.9x2.5_344.cam.h1.0000-01-05-00000.nc AQUA_ZM_00_1.9x2.5_344.cam.h1.0000-01-06-00000.nc AQUA_ZM_00_1.9x2.5_344.cam.h1.0000-01-07-00000.nc AQUA_ZM_00_1.9x2.5_344.cam.h1.0000-01-08-00000.nc AQUA_ZM_00_1.9x2.5_344.cam.h1.0000-01-09-00000.nc AQUA_ZM_00_1.9x2.5_344.cam.h1.0000-01-10-00000.nc ...
Most of the time I manipulate and analyze data with NCL, which has this really great function, addfiles(), that concatenates a variables across hundreds of files. It’s really fast and works great… until you get about 3 years worth of files. At that point I was getting a segmentation fault because there were too many files!
Concatenating NetCDF files with CDO is pretty straightforward:
cdo cat *.nc outfile.nc
This will include all variables, and concatenate the time dimension.
But what if you only want to strip off one variable? That by itself would be something like:
cdo -select,name=PRECC ifile.nc ofile.nc
And what if you also want to limit the domain to save space? That is generally done:
cdo -sellonlatbox,0,360,-40,40 ifile.nc ofile.nc
We can combine these two request as follows:
cdo -sellonlatbox,0,360,-40,40 -select,name=PRECC ifile.nc ofile.nc
Notice here that the order matters. I don’t quite understand why, but switching the order of the operators here gives me a segmentation fault.
Now, to get what we really want. I found that I don’t need the “cat” operator if I’m selecting a specific variable:
cdo -select,name=PRECC AQUA*.nc ofile.nc
However, when I try a similar request with the -sellonlatbox option I get an error:
cdo -sellonlatbox,0,360,-40,40 AQUA*.nc ofile.nc >cdo sellonlatbox (Abort): Too many streams! Operator needs 1 input and 1 output streams.
So in this case, it appears you need two steps if you really want to reduce the area of the data.