Meteorology education - Shortfalls as I see them and a possible solution

by

Chuck Doswell

 


Posted: 13 March 2006 Updated: whenever

As usual, this is my opinion and feedback, positive or negative, is always of interest.


1. Introduction

My goodness, you say - yet another rant about something that bothers Chuck Doswell. Is there no end to these? Perhaps not. Be that as it may, I've been much more involved with students at the University of Oklahoma since I "retired" than ever before. I have funded research grants and this necessarily involves dealing with students. I enjoy interacting with students, but in the process, I've detected some disturbing trends within their ranks. Not that I think students are somehow going bad - no, it's more along the lines that we educators seem not to be doing our jobs properly. At least, we are failing with regard to how I think that educational responsibility should be discharged. I've written elsewhere about my thoughts regarding how students should approach their undergraduate and their graduate studies. This rant is addressed to my academic colleagues - a challenge to them if they actually care about how well-prepared their graduates are going to be when they receive their diplomas and go out into the world to seek their career goals.

Of course, if students do happen to read this, they should recognize that if their academic mentors are not preparing them properly, then those students may have to accept a larger share of the responsibility for their education. That is, they should take it upon themselves, if necessary, to make up for what their faculty mentors are failing to do.

 

2. Computer programming - An essential skill set for meteorologists

First of all, some background is helpful here, or at least I hope so. When I was an undergraduate, I began to realize that learning how to program computers was virtually certain to be a skill I needed. No advisor or any other "adult" made this suggestion to me. I figured this out on my own. Realize that this was the mid-1960s. Computers that had a few thousand bytes of memory and perhaps a disk drive that had a capacity of at most a few megabytes, filled entire floors in a campus building, required special "clean" rooms, had an attendant staff that perhaps numbered scores of real people, and we got one "turn-around" per day. You entered programs on punched cards, submitted them to the high priests (the staff) at some time during the day, and you found out how your program worked out the next day! I learned FORTRAN as my first programming language, which had to be compiled (using a so-called "compiler" program that converted my FORTRAN code into executable binary code) first. Once it compiled successfully - usually only after several unsuccessful attempts - then we could concentrate on getting the program to work properly. If it seemed finally to work as intended, what we obtained wasn't typically useful output. Instead, what we received was numbers for output, printed on sheets of wide, pinfolded computer paper. If we wanted a graph or some other visual product, we usually did that by hand from the printed output. This was incredibly primitive by today's standards, but nevertheless, I was excited by the potential associated with what computers could do for me. I could obtain solutions in detail from mathematical equations, I could do statistical calculations, I could take simple data and perform calculations on the data to produce complex products that would help me understand the quantitative implications of those equations. It was like magic, and for a time I even considered switching careers to programming.

What's important to understand is the role that computers play in modern meteorology. Virtually everything we now know and understand about the atmosphere is a direct outgrowth of being able to solve the relevant mathematical problems using numerical methods run on computers. In today's world, of course, kids grow up using computers - but what most of them are still unable to do is to program those same computers. This is not the place for a full discussion of what research is all about (see my discussion of how science works), but it's important to understand that scientific research necessarily entails standing on the frontiers of what we already understand, and then doing something to move those frontiers into uncharted territory. Since it's uncharted territory, there's no "road map" into useful new concepts - it's up to the researcher to create that map from the exploration. Thus, there may not to be any "canned" program to do what the researcher has in mind. Some pre-existing software system might be available to some some part of what is needed, but the essence of research involves the likelihood that the research needs to write his/her own computer code to accomplish precisely what needs to be done. It might later become routine to do such things, and code may eventually become available to do whatever that is, but if new ideas are to be incorporated into readily-available software, someone has to do it first. This is the essence of what being a researcher is about. In the era of computers, someone who can't program is the functional equivalent of someone who drives a car but has so little understanding of how a car works, they can't fix it if something goes wrong.

To be a scientist virtually requires you have the capability to do your own computer programming - to solve problems of your own choosing, rather than to use the programming skills of someone else as a sort of "black box". To me, this is so manifestly evident, and was so even as long ago as the mid-1960s, causing me to accept the responsibility to learn programming before it was obvious to anyone that a meteorologist without programming skills was essentially a cripple - someone without legs trying to run a 4-minute mile.

Has the need for programming skills diminished in the science of meteorology with time? No - if anything, the need for such skills is increasing every year that passes!! Nevertheless, I consistently encounter students, both undergraduate and graduate, who have only marginal programming skills, if any. This is both profoundly disturbing and astonishing. If students aren't getting the message that programming skills are mandatory for a career in meteorology, then both the students and their mentors are at fault. This essay grows out of my frustration over this problem

 

3. Problem-solving and programming

Anyone devoting any time at all to meteorology will discover quickly that this is a science that depends heavily on mathematics and physics. In its early history, meteorology was a scientific backwater. Why? Because the equations that describe the physical principles associated with the science of meteorology are, for the most part, unsolvable. Analytic, closed-form solutions to the governing equations of meteorology don't exist, save for a tiny set of highly restricted circumstances. In other words, the mathematics of meteorology are not generally solvable by purely mathematical methods. Only by making some severely restrictive assumptions (which generally are unrealistic) can purely mathematical solutions be obtained.

Therefore, until the advent of computers, our capability to understand meteorology was restricted to purely empirical relationships and a tiny set of highly idealized problems for which mathematical solutions could be obtained. A very good argument can be made that meteorology as a problem in physics simply could not be solved without computer programming. Most modern notions of dynamic meteorology trace their origins to the ability to solve the pertinent equations by means of computer programs.

Furthermore, it goes well beyond being able to solve the mathematical equations describing the physical principles underlying meteorology. The quantitative assessment of data also entails being able to do complex operations on those data. Yes, many "canned" programs have already been written to manipulate meteorological data, but suppose you want to try something new? And how well do you understand those "canned" programs, which to many are "black boxes" about which the users may know little or nothing. How dangerous is it to trust what goes on inside such a "canned" program? I assert that it can be very dangerous - a proper application of some pre-existing code might depend on some assumption that might or might not apply to the data you wish to process. Better to know what is going on within some canned program than to choose to remain ignorant, trusting that it's doing what you want it to do.

If you program something, I assert that you necessarily must understand it at a much deeper level than if you simply are using it as a "black box". Programming some procedure always requires that you have a fundamental grasp of that procedure's logic. Anything less results in either faulty code, or perhaps even an inability to complete the effort. Programming always exposes any weaknesses in your understanding of the logic associated with some procedure. Being able to program a procedure always carries with it a more profound knowledge than simply getting an "A" in some course, or passing a test. Programming is demanding and what it demands is in fact very important to one's success as a meteorologist.

It seems evidently obvious to me that an education in meteorology must make it absolutely clear that computer programming is a skill that students need. I'm astounded that more homework problems and classroom exercises in problem-solving do not include requiring all meteorology students to develop and use their computer programming skills. They should be required to write computer programs to solve problems from the junior level of undergraduate education, and first courses in programming should be taken no later than the sophomore year. If it's appropriate to expect all students to have a laptop computer these days and to be Internet-capable, it's just as appropriate to expect meteorology students to be able not just to use computer programs, but to write their own code to solve problems. Anything less is doing our students a disservice, and any student who thinks they can "skate by" without programming skills needs to be disabused of this misconception as soon as possible.

I find it hard to understand why anyone would resist such pressure. Being able to answer your own questions about the principles of meteorology seems like an incredibly exciting capability that any serious student should accept willingly. If you want to understand the subject, it should be at the deepest possible level. That deepest level, in this day and age, is only accessible to those who are reasonably skilled at computer programming - period. And any educator worthy of the title should be eager to reinforce that message to incoming students.

4. Numerical weather prediction - a modern requirement

Again, some personal history might be worthwhile in making the point clear. When I was a first year graduate student, I took a course in "Numerical Weather Prediction" (NWP) - taught by the late Prof. Rex Inman at the University of Oklahoma. This was a one-semester course, during which we were expected to program our own individual versions of a barotropic forecast model. Such a model is a very simple NWP method that reflects the pioneering efforts done by our scientific predecessors in the years following World War II. The very first operational forecast models were barotropic models, so we were effectively reproducing in the late 1960s what had been at the very frontiers of meteorological capabilities only 20 years earlier. We were expected to do what people only 20 years earlier had pioneered.

In one semester, essentially everyone in my class did precisely what was expected of them, using a relatively primitive computer, that had a few hundred Kbytes of memory and perhaps 10 Mbytes of disk space. We began with rawinsonde data, produced an objective analysis of that data, did some primitive "data assimilation," developed our own code to implement an "equivalent barotropic" forecast model, and then produced graphical output from the 12-h forecast we obtained using the model. This required us to consider the following topics:

  1. Use of map projections, including how the so-called "map scale factor" worked in the context of our forecast model
  2. Setting up a forecast and analysis grid on the image plane of a map projection
  3. Numerical stability of the forecast model
  4. Objective analysis of meteorological fields
  5. Dynamical relationships among the standard meteorological variables as described by the model
  6. Interpolation procedures
  7. Numerical integration of partial differential equations
  8. How to fit a complex calculation involving several steps within the constraints of a computer's capabilities
  9. Data input formats
  10. Data output formats compatible with how those data are going to be used
  11. Displaying output data in a useful form
  12. Basic principles of forecasting, whether objective or subjective
  13. Scale analysis, including an understanding of why a simple barotropic model can be useful

There are other topics that might be useful to learn regarding NWP today that were unknown or at most only topics for research at the time we were taking this course. In what follows, I will include what I believe would be important additional topics in an NWP course taught in the current era - such a course might be two semesters, rather than one, but the importance of NWP cannot be overemphasized, for reasons to be offered next.

 

5. NWP as a "capstone" of a meteorological education

If there was anything frustrating to me about the academic process as offered to me, it was that courses were taught as self-contained "islands" - little boxes that contained what the instructor thought was the content, and any connections to other courses were never mentioned. Thus, for many students, meteorological education is seen as a series of disconnected topics and it's very uncommon for any instructor to make any effort to pull the disparate pieces together. If the students make those connections, it's typically on their own initiative rather than by the design and plan of the course content.

My experience in the one-semester course in NWP taught by Rex Inman was that he succeeded brilliantly in tying together the components of most, if not all, of the courses I had taken up to that point. Suddenly, the mathematics, physics, meteorology, and programming courses I had taken as quasi-independent topics came together in such a way as to produce the very exciting prospect of writing my own forecast model. At the end of his NWP course, I had a very real and useful understanding of all these topics, pieced together in such a way that I could never look at the product of a forecast model in the same, ignorant way I had been seeing them. I knew, because I had programmed similar procedures, what was involved in this ultimate expression of our meteorological knowledge - a computer-based NWP model.

When you've programmed something, as I've already said, you know that material at a far more intimate level than what you get from the process of passing an exam. Your understanding is much deeper than simply being able to regurgitate the material on demand. You've written step-by-step algorithms to accomplish a goal - you know the workings of those algorithms at the deepest possible level. They aren't some mysterious "black box" you use without any idea of how it works and the notion of accepting any analysis as a "black box" becomes unsatisfactory to you.

The beauty of NWP is that encapsulates all that we currently know about the atmosphere, and about how to coax that knowledge from raw observations. It ties together all the topics that I've enumerated, above, in a way that has to be compellingly clear to those who have successfully implemented an NWP model. Even if you have no plans to become a researcher or an NWP modeler, it is manifestly clear in the present age that you will have to consider what NWP models have to offer. You have to understand the inner workings of those models to have any hope of being able to use them for some purpose, if nothing else. NWP is the apex of our meteorological understanding.

The more complex the model, beyond the simple equivalent barotropic model we wrote during my coursework in the late 1960s, the more principles a student needs to understand:

  1. Baroclinic models and QG theory
  2. Primitive equation models
  3. Spectral versus gridpoint models
  4. Modern concepts of data assimilation and why it's important
  5. Advanced numerical methods
  6. Ensemble forecasting and stochastic-dynamic modeling
  7. Model verification and statistical analysis
  8. Model post-processing methods for forecasting, including MOS
  9. Physical processes, such as boundary layers, radiation balances, microphysics, etc.

It seems to me that a 2-semester course of NWP could serve as not only an opportunity to develop a student's programming skills, but also an ideal venue for pulling together many of the disparate pieces of their education in their senior year. This is where the individual components come together and the process requires some considerable care to get it right. You're forced to develop those individual components in the context of the model and if you don't understand each component well enough to code it for your model, you'll be forced to learn that material. This can be stimulating, as it takes the mystery out of the process. For example, I first learned about objective analysis in Rex Inman's NWP course, and I'm still exploring that topic right up to the present. NWP not only connects all the topics of one's academic preparation, it also can stimulate a student to undertake an entire lifetime's worth of challenging research. I see it as the ideal "capstone" to an undergraduate education. It should be a mandatory course for every undergraduate meteorology program, and if someone comes in with a Bachelor's degree from another field (such as physics or engineering), NWP is the ideal course to give such a student a comprehensive and practical understanding of meteorology. No matter what the intentions of a student are for a career beyond the Bachelor's degree, NWP contains valuable content for any meteorology major. That students are getting graduate diplomas without ever having taken NWP seems outrageous and inexplicable to me.

Furthermore, it would ramp up the computer programming skills of students to the point where they would routinely expect to be able to solve problems by writing programs to do quantitative analysis. It's inexcusable for us to let students graduate at even the undergraduate meteorology level without being competent computer programmers - not just computer users.