Category Archives: Computer Tips and Tricks

A collection of tips and tricks I’ve picked up over the years for dealing with large files and complex numerical models.

CESM: Apparent Errors from shr_sys_flush()

I’ve been struggling with a model error for over a week now, and it all seemed to revolve around this Fortran subroutine “shr_sys_flush()”. The error usually showed up in the log file as a “floating point exception”, but other times I got a “segmentation fault”. I had an epiphany this afternoon that will hopefully help other people with similar errors. Continue reading

NCL: A Function for Calculating Effective Static Stability

In my last post I discussed a paper that described the theory of effective static stability (O’Gorman 2011). Dr. O’Gorman posted a MATLAB script for calculating effective static stability, but since I work with NCL, I wanted convert his code into an NCL function. The original MATLAB code only worked on one profile at a time, but the function below “should” work for a multi-dimensional array, but I won’t guarantee that it will work for any array size. Continue reading

NCL: A substitute for pointers when dealing with arrays of different sizes

A “pointer”, in computer science terms, is an object that references a place in memory. This differs from a variable or an array, because a pointer does not have a type or size that changes. This allows some useful things that are impossible with regular arrays, such as having an array of pointers where each element points to an array of a different size. I recently found a nifty trick in NCL that provide a similar advantage.

Continue reading

NCL: Averaging in blocks (i.e. converting to longer time steps)

I often find myself needing to compare datasets with different temporal resolution, like 3-hourly and 6-hourly. This is not a trivial thing to do on the fly, and storing redundant datasets for these occasions is wasteful. So I made an NCL function that averages the leftmost dimension into blocks, effectively degrading the time resolution by whatever factor I want. Continue reading

NCL: Cropping the white space from a PNG file

The NCAR Command Language (NCL) has a great set of built-in functions, with outstanding documentation, but there are many times when you can speed up your work flow by creating your own library of functions. I’ve been slowly building a library of custom NCL functions that have been really useful over the years. There’s no mechanism for me to get them added to the standard NCL library so I thought the next best thing would to just publish them online and hope other people will find them useful as well. Continue reading

CDO: Bulk Diagnosis of Model Output

Have you ever wanted a simple way to output global average values for each year from a series of monthly climate model output files? I have found myself in this situation several times, where I needed to check on various aspects of a climate model simulation (i.e. has the ocean temperature stabilized?). However, the data is usually distributed over many files, which makes this task difficult. Continue reading

CDO vs NCO

When I need to manipulate large data files (i.e. 100’s of GBs or TBs), I usually write a script in NCL or Python to do the job. Recently I’ve realized that this is often a huge waste of time! The problem is that my approach requires the task to be broken down into chunks, which slows the process by several orders of magnitude due to the computational overhead that results from iterations, as well as my own time to write the code.

A much faster and cleaner way to manipulate large datasets is to use command line operators that are specifically designed for climate datasets.Two libraries have been independently developed to do this, specifically: Continue reading

Screen Recordings in Mac OSX

I accidentally stumbled on this little gold nugget of a post that explains how the OSX operating system has come with a screen recorder since 2009!!!! It’s part of QuickTime. I don’t need this feature very often, but when I do I’m never able to find a quick, easy way to do it. This is a great way to record lectures with the built-in mic of a laptop! Continue reading