Severe Weather Warnings - An Overview With a Look To the Future


by

Chuck Doswell



Posted:  20 February 2013  Updated:  whenever -

This is in response to some recent questions I've been asked, so I'm providing a summary here.  As usual, this is purely my own opinion and represents nothing that has been formally reviewed/vetted by anyone.  Comments are welcome if you're willing to have them posted here, along with my responses.  Send me an email to cdoswell & earthlink.net (click on the email link or cut and paste, replacing _&_ with @) with your comment(s).

Introduction

The first order of business with respect to severe weather warnings is to recognize that with all its problems and challenges, the existing system for severe weather warnings (of all sorts, including hurricanes, tornadoes, winter weather, flash floods, etc.) is responsible for having saved tens of thousands of lives (by my estimate) since the modern process began in the spring of 1952, with the inception of the Severe Local Storms Unit (SELS) - the forerunner of today's Storm Prediction Center (SPC) in Norman, OK.  I'm going to focus on severe convective storm hazards herein - primarily tornadoes.  I've said a lot of the topics mentioned here in other essays, blogs, etc. - this simply puts all of the things I've said previously in the context of an overview.  Once the Weather Bureau (forerunner of the National Weather Service - NWS) understood that the mention of the word "tornado" in a forecast would not initiate a dangerous panic among the populace, the way was clear to begin to apply the science of meteorology and new observing technology (mainly, weather radar) to the task of providing life-saving information to people in threatened areas, as well as deploying storm spotters to supplement the radar.

Thus, before we start tinkering with the warning system, we must keep in mind a simple principle often applied in the health care profession:  First, do no harm!  If we begin to mess around with the warning system in the absence of clear evidence that our changes will cause no harm, we run the risk of damaging the credibiity of our warnings beyond repair.

The warning system didn't evolve in any orderly, systematic fashion.  Rather, a series of top-down mandates were issued, starting in 1952, to commence tornado forecasting, so forecasters had to put together the elements of the system in response to those mandates in the absence of any guidance and supporting information on which to base their ad hoc decisions.  No one has ever done any systematic analysis of how best to do this critically important task.  It begs the question:  If you could wipe it all away and re-design it from the ground up, what would the warning system look like?  It's highly unlikely anyone will ever do the work to re-create a warning system starting from nothing - it will always have to fit within certain constraints created by where we are now.  But suppose we had that privilege - how might we go about a systematic program to develop the most effective warning system possible.

Step 1 - What is a warning system trying to accomplish?

Keep in mind that a warning is a forecast - generally, a short-range forecast (on the order of an hour or less) for convective storm activity.  Because it's a forecast, it's inherently uncertain.  Convective storm processes can change substantially in five min or less.  A tornado can form from nothing in that short interval, so forecasting convective storms is a challenging task.  I believe the goal of a severe weather warning is to provide effective severe weather information to forecast users of all sorts, in order to help them make appropriate decisions when confronted with a potential threat of severe weather. Broadly speaking, it's my impression that most people have quite limited understanding of the quantitative risks they confront.  People make choices based on their perception of risks, not on the actual risk level.  We kill 40,000 American people per year in traffic, but no one is particularly fearful of driving, per se.  More than 10,000 people die in the US from food poisoning, but most people have no fear of eating.  In the worst years, fewer than 1,000 people are killed by tornadoes annually in the USA - in recent decades, typically fewer than 100.  Although the tornado fatality risk is small, someone dies in a tornado every year.  It's possible to prepare for the tornado threat without giving up much.  People always have to make choices - no decision is always still a decision, of sorts!

For instance, when most homes (even in Oklahoma) are built without a tornado-resistant shelter, this suggests that the perception of tornado risk is low enough to choose not to spend that money on a shelter, preferring instead that that investment should go toward a jacuzzi, or fancy kitchen cabinets, or a swimming pool.  Given that the perception of risk even in Oklahoma is that low, it seems unlikely that a few minor wording changes in our warnings are going to make them dramatically more effective!  Imagine the perception of risk in New England, or the intermountain west and what that might imply about public preparations for severe weather.  The choice of whether or not to do something to prepare for tornadoes depends on that very perception of the risk.  Obviously, we meteorologists believe we have the public's best interests at heart with our warning efforts, but if "the public" (far from a monolithic block of people with identical risk perceptions and level of commitment to being prepared to deal with severe storms) has a dramtically different set of priorities from those of meteorologists, some careful thinking and well-conceived research are needed to do something about that perception gap.

I believe strongly that we need the help many social science-related disciplines:  communication specialists, sociologists, psychologists, economists, etc.  We need hard information about public attitudes and perspectives.  If we make changes to the warning system without compelling evidence based on solid data that we're changing it in a way that really will allow our warnings to be more effective, we risk doing harm to the existing system.

Step 2 - What is good and bad about the existing system?

Any attempt to change the warning system necessarily should begin with an assessment of public perceptions regarding our existing warning system.  What are the most effective aspects of the present warning system?  What are the problems with that system?  There may be regional variations in public perception, of course.  And those perceptions change all the time, so this isn't a task we can do once and it's done forever.  A commitment to use the social sciences to guide how our warning system should be constructed involves a substantial, long-term commitment of resources.  I don't believe many in the NWS appreciate the magnitude of the commitment to the involvement of the social sciences they appear to be willing to make.

From where I sit, an evaluation of the existing warning system is a massive task that remains more or less undone in any scientifically substantive way.  No one knows very much about public perceptions because the work has only begun!  What we typically have in abundance are comments from bureaucrats and meteorologists saying words along the lines of, "What the public wants is ...!"  These words actually mean, "What I think the public wants is ...!"  There's precious little evidence to back up most such statements.  We have the unvalidated opinion of someone who knows essentially nothing about what "the public" actually wants - it's more based on that person's anecdotal experience than carefully-done studies.  In other words, such comments have little substantive value!  I've tried (and failed) to get a proposal funded to investigate these issues, with the collaboration of a social scientist.  Multidisciplinary research is still not supported very strongly - it's difficult to find potential collaborators outside of your own discipline and put together a plan for a project of mutual interest.

Let's assume, nevertheless, the research has been done and we can say with some confidence that we know just what is working and not working with the existing system.  Now we're ready to consider ...

Step 3 - How do we go about making the warning system be more effective at doing what we want it to?

Producing accurate severe weather warning information is not that easy, as I've suggested, but let's assume for the moment that the information we hope to provide already exists at some moment in a weather office.  In order for that information to be effective, the following conditions must be met:
  1. The users must receive the information
  2. The users must understand the information
  3. The users must know what to do with the information
  4. The users must believe the information
  5. The users must be able to take effective action
  6. The users must respond by taking that action when the potential threat materializes
However we might define perfection, even a perfect forecast/warning will be of no value if any one of these conditions isn't met.  There are many pitfalls along this path, even if the severe weather forecasts do their job perfectly.  When users don't hear the warning, all the rest is irrelevant - the warning is useless to them.  If they hear it but don't understand what it's saying - the warning is useless to them.  If they hear it, understand it, but don't know how to use that information - the warning is useless to them.  If they hear it, understand it, know what to do with it, but don't believe the information (for instance, because they perceive such warnings are predominantly false alarms) - the warning is useless to them.  If they hear it, understand it, know what to do with it, believe it, but there's nothing they can do about that information (for instance, because they have no effective place to take shelter) - the warning is useless to them.  If all the conditions are met except they choose not to respond (for whatever reason) - the warning is useless to them.

Make no bones about it, the meteorological challenge to producing a severe weather warning is not a trivial thing.  But an even bigger problem is associated with what happens when the forecast is disseminated.  The effectiveness of that warning, once it has been issued, is mostly out of our hands.  We meteorologists aren't trained in communication, sociology, psychology, etc.  Since the goal of our warning system it to provide information, that information requires
  1. Mechanisms of dissemination to allow users to receive the warning
  2. Public education to help users understand what we are trying to say in our warnings
  3. Public education to help users recognize the appropriate things to do
  4. Public education to understand the limitations of our ability to forecast severe weather
  5. Public education to help users prepare for severe weather by giving themselves via options
  6. Public education to help users use the information to take action
Based on the preceding, it seems to me we have a strong obligation to be much more effective in our public outreach efforts -  to educate the public about all these aspects of the risks they confront from severe weather beforethey're forced to deal with a severe weather event.  We can't make up for that ignorance in the 30 minutes (or less!) that might separate the issuance of a warning from its impact on a community.  To a large extent, our ability to produce effective warnings depends heavily on what we've done beforehand - prior to the time when dangerous convective storms threaten our users.   We have to make certain everyone is on board with what we can provide them and how to prepare/respond.  No amount of tinkering with the wording of warnings can fill that gap!

We need broad-based surveys, done by people who actually know something about how to construct and carry out a proper survey, working with meteorologists who can explain what we do and, particularly, the limitations on what we can provide.  Surveys need to be done to assess the potential impact of proposed changes before we even consider implemented them in public forecasts.  We need to be confident that any proposed changes meet the preceding conditions on the effectiveness of the warnings. 

Once we have some idea of what changes seem to make sense to implement, then we should do experiments based on the notions that have survived this rigorous pre-testing.  Depending on the outcome of those experiments, we should do more surveys to see if they've indeed been successful  If, in the process, we discover a problem with a a proposed change, then we need to revise it or eliminate it.  Only those changes that prove to be effective should be implemented operationally.

And the need for continuing surveys and re-evaluation of NWS warning products should be evident.  The landscape in which warning products are issued is constantly changing.  It's affected by new technology, new science, new economic realities, social change, and so on.  As noted, there will be a sustantial investment required if there's to be an enduring meaningful collaboration between meteorology and social science.

Step 4 - How do we best incorporate uncertainty into severe weather warnings?

To repeat, warnings are forecasts.  All forecasts are uncertain.  Therefore, all warnings are uncertain.   This can't be avoided and should not be swept under the rug or ignored.  Adding uncertaintly information to warnings is a dramatic change from the existing system.  Forecast uncertainty drive forecasters directly to the classic dilemma of an asymmetric penalty:  no one dies from an event that didn't happen!  This asymmetry strongly encourages overforecasting using a dichotomous (yes/no - binary) forecast product.  Thus, the severe weather warnings all show a heavy overforecasting bias:  a preponderance of false alarms.  With probablistic forecasting, it should be possible to achieve unbiased warnings.  Furthermore, to fail to express uncertainty is to withhold information from our users.  We meteorologists are acutely aware of our uncetainty and, when properly calibrated, are quite good at estimating it quantitatively in terms of probability.  Keeping this to ourselves is doing our users a huge disservice.

The challenge we all confront is to be able to express that uncetainty in a way that is understandable, believable, and effective in helping people make decisions about how they should respond.   Although it's well-known that probability is the proper language of uncertainty, there may be ways to express it that don't use the dreaded "p-word".  Or we might produce a forecast product that caters to all potential users of that information, not just the lowest common denominator.   I've discussed this topic at some length elsewhere, but I admit freely I have no clue how best to accomplish this goal in the most effective way for severe weather warnings.  

The simple reality is that severe convective storm warnings should be able to express variable threat levels.  Not all situations constitute an equal threat.  In order for a specific situation to generate a warning, there should be some systematic way to decide some minimum threat probability that would trigger a warning.  Once that threshold is achieved, then the warning should be able to convey the warning forecaster's confidence that a meteorological threat exists in a specific region for a specific time interval.  I make no claim to anything more than some vague ideas about how best to achieve this goal from the standpoint of all the conditions enumerated above (in Step 3).  My education and training is in meteorology, not risk communication.  But as a meteorologist, the need to be able to express uncertainty is not negotiable.  It's necessary and inevitable if we wish to convey accurately and comprehensively what we think about the weather!  See the links above for some discussion.

There have been a handful of notable successes using the phrasing "tornado emergency" to indicate a potential high-impact event.  Less publicized are the many times "tornado emergency" has been used without a high-impact event actually occurring.  A few notable successes constitute only anecdotal evidence that forecasters can discriminate potential high-impact events from lesser impact situations.  Any experiment done without a thorough evaluation of our aggregate ability to forecast them is premature, at best, and certainly ill-advised.

Ideas about how graded threat levels might be done should be tested in surveys to gauge public reaction to them.  Depending on the survey results, different ways to express uncertainty  might need to be tried.  Those potential changes would need to be tested ... and, of course, the testing, evaluation, and revision should go on indefinitely!  It's a virtual certainty there's no single answer that will fit all possible future situations now or in the future.

Discussion

I've not really spent much time  on how best to do the meteorology associated with severe storm warnings.  With all of its imperfections, the existing warning system works pretty well, especially for long-track, violent tornadoes associated with supercells.  Since these account for a disproportionate share of the annual tornado fatlalites, this aspect of the system seems to be working pretty well, for the most part.  We certainly have work to do to improve our warnings for lesser events but this is not very pressing, given the realtively low fatality counts associated with this relatively poorly forecast class of tornadoes.

As I see it, the most important meteorological issue associated with severe storm warnings is to quantify the uncertainty.  Education and training can help with some of this.  Proper calibration of severe weather threat probabilities is a challenge precisely because severe convective storms are rare events.  Calibration takes experience and it takes years of experience to do it really well.  I won't dwell on training and education issues, but they constitute a problem within the NWS, which is an organization not committed to meaningful training and which assumes zero responsibility for education.  Under these circumstances, forecaster skill at severe weather warning will continue to span a wide range, with a few superstars, a few superduds that need to be flushed, and a vast middle ground of variable forecasting ability.  Product consistency is a known issue within the NWS, for convective storm events and others, as well.

The meteorological issues aren't trivial, but I don't see them as the major barrier to creating more effective warnings.  Unfortunately, when you convene meteorologists to address the issue of forecast value, they tend always to drift toward what they know:  meteorology.  We shouldn't shut out the social sciences, but rather get them involved immediately.