Are you ready for the defining moment of your forecasting career?

by

Charles A. Doswell III


Posted: 08 March 2005 Updated: 26 May 2011: some clarification based on reader input and fixed broken links

This has been inspired by hearing a presentation by James Spann, a weather broadcaster in Alabama - and a very classy man. As always, this is my opinion and comments are invited. Contact me at cdoswell # earthlink.net [Use the email hyperlink or cut and paste, replacing " # " with "@"].


Introduction

The notion of the "defining moment" of a forecasting career began to dawn on me several years ago, talking with forecasters about certain key moments which confronted them, usually in the context of a tornado outbreak or a killer flash flood event. The basic concept is simple: in the career of a weather forecaster, most days are characterized by a boring banality. The forecast goes out, the warnings are issued, the bureaucracy's needs are dealt with, and the days have a kind of numbing sameness. This doesn't mean that all days are the same, but it does mean that most come and go without having much of an effect on the life of a forecaster. This is simply a result of statistics: the statistics of rare events. As I've discussed elsewhere, actually experiencing a major event for oneself is unlikely. Even in the heart of tornado country, a thousand years might pass between repetitions of an F5 tornado.

Forecasters usually work rotating shifts, with each position having a team of five forecasters to cover all the shifts (see here for a related discussion in the section about "common issues"). Therefore, a forecaster works roughly 1/5 of the week, so that if some important event happens during a week, there's only about a 20% probability that it will occur during the shift of a particular individual (actually, of course, it's less than 20% because no forecaster works every week, ignoring the times a forecaster gets called in for overtime during severe weather). If the annual climatological probability of having an extreme event is one chance in 1000 (a probability of 0.001 or 0.1%) in any given year, then in a 25-year career, that means approximately (but not exactly - see the August 1999 update here for more details) 25 chances in 1000, or approximately 2.5% probability of any forecaster having worked such an event during that period. But if a particular forecaster only works one day out of five, that lowers the chances of that forecaster actually working on shift during that particular extreme event over the course of a 25-year career to 0.5% probability.

To have as much as a 50% probability that at least one forecaster out of some randomly-selected group with 25 years of experience will actually have worked a particular type of extreme event - with a probability for that 25 years of 0.5% - the group would have to include more than 100 forecasters. There are, of course, several types of extreme event that could occur over the course of a career, which would increase the probability of working some type of rare event. Nevertheless, all in all, in the course of most forecasterÕs careers, whatever the numbers actually are, it remains quite likely that they will not have worked a truly extreme event.

In spite of those long odds -- each year, a few forecasters do have the rare experience of working an extreme event. What this usually means for those few individuals is that those events become the "defining event" of their careers. Their performance during that one big event of their careers is their big chance for notoriety, either positive when their performance is good, or negative when their performance is poor. In extreme events, it often is the case that fatalities occur - a poor performance in a fatality-producing extreme event can become a huge burden for that forecaster to bear. Even if the performance is exemplary, living with the reality of the consequences of that extreme event can be difficult, so a bad performance is often devastating to the forecaster. I know of examples of both types of performance, although I have no intention of naming names.

The consequences of an extreme event provide the basis for a forecaster's defining moment, then. Those consequences can be enormous, literally involving life and death, or the possibility of permanently crippling injuries, for the victims of such extreme events. Forecasters have to recognize that they hold the power to make a large difference in the lives of their forecast and warning users. There's a human face to the consequences of a good or bad forecast. Who would want to have to live with the consequences of fatalities in a case where the forecaster knows that they did a poor job? Would it not be a matter of considerable satisfaction to know that the job done in a "defining moment" forecast or warning was directly responsible for saving lives?

Given the relatively random processes involved, there is no way to know if such a moment will actually occur during any individualÕs career, or when during that career the defining moment will occur. It can be early or late within that individualÕs career - but the most likely outcome will be that it will not occur at all!

Practice

Given that working an extreme event is unlikely even over the course of a 25-year career, how much practice at it can a forecasters have? Obviously, it's very unlikely that it will happen more than once, so forecasters generally have only one chance to show what they can do during an extreme event. It's as if you're given just a single swing of the bat during a baseball game. You can only win the game or lose it with that swing - no practice swings are even allowed.

How can you possibly prepare for such an event? What can you do to prepare for the most important event of your career, when you don't have any idea when or even if it will occur?

This is the dilemma faced by forecasters, whether they realize it or not. Forecasting isn't just some game we play with verification statistics and computers - this "game" can have life or death consequences. Forecasters need to accept this if they're to have much of a chance to be able to feel good about the defining moment of their careers.

So what can you do to prepare? I have several thoughts about this.

Working style on slow days

Many forecast shifts are associated with slow, boring weather. Not much is going on - given that you have many tasks to do, it's tempting to use shift time on slow days to work on research projects, do computerized "distance learning" training exercises, develop computer programs, fill out bureaucratic paperwork, etc. Although I understand the pressures to use this seemingly empty time to do such non-forecasting tasks, I'm trying to suggest here that this isn't necessarily the best way to use that time if you want to be ready for your defining moment, should it ever come to you.

Every period of boring weather is punctuated by a transition to something far more interesting and challenging. How are you going to recognize and anticipate that change if you're not following the weather? Are you going to depend on the models to do that for you? I assert that if you live and die by the models - well, you're likely to die a lot! In the case of your defining moment, it would only take one time to regret placing such dependence on the models, although I suppose it would be a convenient rationalization to shift the blame to the models. It seems to me that the best way to anticipate the change to a more challenging weather situation is to stay on the "met watch" and keep abreast of what's actually happening, not just what is forecast in the fantasy world of the models.

One point that may need some clarification is that on so-called "boring" weather days (e.g., days without strong convective mesosystems, or whatever), there are still atmospheric processes ongoing. The skills to which I'm referring aren't necessarily those associated with the specific processes that characterize a day that might constitute a "defining moment" (e.g., convective mesosystems or frontogenesis or whatever). Rather, it's your general ability to do diagnosis that is critical. Your general diagnostic skills depend on being able to look at observations and combine that with your understanding of the observation systems producing those observations and with your knowledge of atmospheric processes to synthesize a conceptual picture of what's going on that day. Among other things, this includes the ability to recognize the distinction between a bad observation and an observation that is your first indication of an important development about to unfold. This is never wasted time as it permits you the opportunity to improve your forecasts on "boring" days as well as to practice those critical diagnostic skills.

The skills you need on a day when it matters need to be used regularly. What happens to a skill you don't use? You lose it! And the best time to put a 100% effort into using those diagnostic skills is precisely on those days when the weather is boring. During active weather days, you won't have time to practice those skills - you'll be too busy using them! But if you only use them on the days when weather is active, I can virtually guarantee that you'll be rusty and likely won't perform at your best when it matters the most.

In my experience, many busted forecasts on important weather days result from forecasters thinking that today's weather is just like yesterday's, when nothing of significance happened in the weather. As I see it, if you think today's weather is going to be just like yesterday's, then you're already wrong. In some important weather situations, many of which are associated with flash floods (for example), the weather on the day of the event is not night-and-day different from the previous day - rather, the differences can be relatively subtle. If you've not been paying close attention, those subtle differences can result in a surprisingly huge difference in the weather that you might not be able to anticipate. On the day before, nothing of importance happened - on the day of what might turn out to be your defining moment, the weather exploded into strong storms. The small differences were just enough to make that difference.

What's been found in many such cases is that the forecasters who failed to anticipate the possibility of important weather events spent way too much time trying to figure out what's going on as events blow up around them. By the time they figured it out, for important convective weather events, the time to act had passed. People died, the damage was done, and their defining moment was one of tragedy and failure.

Practice with displaced real-time simulators

One option for dealing with the unusual event is to take advantage of previous experiences with similar events. In training exercises, it helps to work the events in simulated real time, to practice doing the things you need to do. However, as useful as this sort of exercise might be, there are two major issues that confront you. One is that if you already know the outcome, the value of the exercise can be lost. In a real situation, you don't know the outcome in advance. It might be nothing or it might be your defining moment. Second, every weather event is unique. The things you needed to do to deal effectively with some past event aren't necessarily the same as those things you'll need to do in real time when your defining moment comes. As I've suggested, you won't necessarily learn what you need by working some past event.

Real time diagnosis

In practicing your diagnostic skills in real time, you have the advantage that you really are seeking clues about the possibilities inherent in the situation at hand. When the weather is benign, you're striving to recognize the early warning signs of a changing situation. It's a way to deal as directly as possible with the reality you may have to face.

In many situations I've seen, forecasters were unable to recognize the significance of their impending defining moment. They weren't thinking about the possibility of it actually becoming an extreme event. Even if they thought it had some potential, they were slow to recognize that this was not an "ordinary" severe weather day - rather, it was the dawning of their defining moment. By the time they figured out that the day was going to be extraordinary, the time to act had passed, the opportunity to do a superb job was missed, the damage was done. I've seen examples of this even when the SPC or the HPC had put out excellent guidance products for the local forecasters. For some reason, those local forecasters failed to recognize the importance of the situation, despite that good guidance. Sometimes, of course, the guidance available to a local forecaster is not all that good. In such cases, it's even more essential for them to be working assiduously at diagnosis, trying to anticipate the potential impact in every forecast shift. When the guidance is good, it's even more inexcusable to be asleep at the switch on the day of your defining moment.

The most important thing, it seems to me, is to put 100% of your attention to the weather on every forecast shift. How would you feel if you've been spending your precious time doing something unrelated to that day's weather when all hell breaks loose around you? This seems to violate the trust that your forecast users have put in you. Do you want to have to live with the consequences? I'm pretty sure I wouldn't.

Thus, I believe that your best strategy for preparing to cope with your defining moment is to put out 100% of your forecasting capacity on every forecast shift, no matter how uneventful it seems likely to be. This way, should it begin to change toward a more portentious situation, youÕre going to be right on top of that change, as it happens. You won't be dependent on the models catching it, either.

Anticipation

So let's suppose that you've been following my advice, and you begin to detect signs of an impending change to a potentially high-impact weather event scenario. Should you immediately write an amended forecast to call attention to this? Not necessarily. The key to responding to changing weather events, it seems to me, is simply to have anticipated them. If you were already thinking about such possibilities, you don't have to spend time trying to figure out what's going on when those first garbled reports come in. Rather, you're prepared to act once it becomes clear that you need to do so. Any forecast has some amount of uncertainty associated with it. No matter how obvious the event might seem when looked at with the benefit of 20-20 hindsight, there always are uncertainties. Your level of confidence in the possibility of some extreme event might not be sufficient at first recognition to be reflected immediately in your official forecasts. But if you've anticipated that such an event is possible, then you're likely to be aware of what changes in the current atmosphere are indicators of an unfolding extreme event. You have some idea of what might signal that a low-probability event is about be realized -- that the low probability is changing toward a high probability. In other words, you have an idea what to look for. Your diagnosis can be focused. On the other hand, if you never imagined that such an event was even remotely possible, it's unlikely you'll be able to recognize the precursor signals. You won't even be looking for them!

It's easy for some forecasters to see imminent disaster in any situation. It's easy for some other forecasters to miss even the most obvious signs of an impending extreme events. What distinguishes a good forecaster from a mediocre one is to be able to make the call the right way when it matters the most. Constantly seeing disaster in every situation is almost as bad as failing to see it at all on the day of one's defining moment.

Acknowlegment. I would again like to thank Mr. James Spann, whose inspirational discussion along similar lines at a symposium held at Mississippi State University motivated me to collect my thoughts and put them down in this essay.

Conclusions

The keys to being prepared to do your best in the unlikely event that you will have to work a "defining moment" include the following: