Meteorology and Users

Chuck Doswell

 


Originally posted: 14 November 1995 Updated: 23 September 2006

* The thoughts contained in this essay are purely my own and do not represent those of the National Severe Storms Laboratory, the Environmental Research Laboratories, or NOAA. My current affiliation is with the Cooperative Institute for Mesoscale Meteorological Studies (also in Norman, OK). You can e-mail me at cdoswell@hoth.gcn.ou.edu


Introduction

Meteorology is in a somewhat unique position with respect to its users. By and large, meteorologists do not generate income; rather, the things that we produce offer opportunities for others to generate income, or for others to avoid spending resources needlessly. This concept is illustrated schematically in Fig.1.

 

 

Figure 1. Schematic illustration illustrating the role of meteorology, which does not generally create income, but it does reduce costs and losses for income generating businesses.

 

Meteorology finds it difficult to point to its value, mostly because its contributions lead to reduced costs and losses, rather than direct income generation. Meteorological efforts don't put cash in the bank, they keep cash in the bank.

This makes it a challenge to account for the financial contributions of meteorology. The typical user doesn't keep a record of money not spent. You don't get calls from farmers telling you how much they save by spraying at the right time and not spraying at the wrong time. The energy distribution companies probably don't maintain a record of what they saved by buying energy (gas, coal, electricity) ahead of peak usage times instead of having to pay extra for energy to deal with situations at the time when their capacity is exceeded. Users of irrigation water don't let us know about their savings when they avoided buying water at times of stress just before it rained because they got a good forecast. And so on and on.

Sure, weather forecasts are wrong some significant percentage of the time. In fact, the more precision we ask of the forecasts, the more likely they are to be wrong. It's more difficult to get the forecast of tonight's minimum temperature within 1 deg F, than to get it within 5 deg F. A quantitative precipitation forecast is more challenging by far than simply to forecast the occurrence of measurable precipitation. It's not trivial to make forecasts of ceilings and visibility accurate to within strict limits. That uncertainty is the name of the game and it's unavoidable. Any meteorologist saying otherwise is being deceptive, or foolish, or both.

However, intelligent users of weather information recognize its limitations in spite of their desires for perfection. Intelligent users tend to develop systematic ways to account for that uncertainty in their decision-making. When big chunks of money are riding on a decision based, at least partly, on a weather forecast it would make sense that the user would be concerned about the methodology of decision-making in the face of uncertainty! Nevertheless, it is disappointing to find users who seem compelled to complain about the forecasts rather than to learn ways to deal with the inevitable uncertainty. A user who insists on a categorical forecast is, in essence, asking the forecaster to make the decisions for him/her, since a decision based on a categorical forecast is a trivial one ... if you believe the forecast. Suppose you insist on a yes/no forecast ... it will/will not rain today, the high will be precisely 86 deg F, the fog will/will not restrict visibility to less than 1/4 mi., etc. - when this is the way forecasts are required to look, the real decision has been off-loaded to the forecaster.

The basic problem with this is that no forecast can be made with 100% certainty. Any user of forecasts finds out about this quickly! Therefore, if you insist on a categorical forecast, but you ask the forecasts to be perfect every time, you will then be unhappy with the product. The forecaster could tell you how certain he/she is of the forecast, of course. Then you wouldn't be in the position of trying to estimate the uncertainty on your own; you'd have additional information to help you make your decision because now you would know to what extent the forecaster is confident of the product. This is the role of probability in weather forecasting: the expression of uncertainty. O.K., this is my standard sermon on probabilistic forecasting. Forecasters should not be responsible for decision-making on behalf of all their users. Rather, decision-making is the user's responsibility, while the forecaster provides meteorological information to help with the decision. Only the user knows his/her particular cost/loss ratio, which is the important factor in using probabilistic forecasts to make decisions. There is substantial scientific literature on decision-making in the face of uncertainty, and an intelligent user should be aware of that literature. Therefore, part of our job ought to include making sure intelligent users have access to information about how to use our products to make decisions.

 

A Major Shortfall in Service to Users

This is a place where I think we meteorologists have been falling down on the job. We need to take on the task of helping our users learn how to use the information we can provide. If users want from us things that we are unable to provide, we need to help them understand how to use what we are able to provide. And we need to stop promising the user more than we can possibly do. This is especially so for the public sector forecasters. In times of decreasing budgets, there is a growing pressure to privatize the functions of the public forecaster in the domains of specialized weather information: agriculture, aviation, marine, etc. The idea is to have our public forecasters put out a "plain vanilla - one size fits all" product for the general public and leave the specialized products for private sector companies. The other function provided by the public weather service would be to provide "warnings" for hazardous weather situations.

In the main, I think I am quite willing to accept this argument. The issuance of a "free" product (i.e., supported by all the taxpayers) for such special purposes as agriculture and aviation interests is at least an arguably inappropriate thing to do. The money saved is by and large saved by only a small number of users. The first line of savings is for the business users like airlines and farmers/ranchers. Of course, they tend to pass on their increased costs to the customers, so it can be said that the general public can benefit by such forecasts. However, the "free" forecast is basically unfair competition for private meteorologists. The "free" forecasts are sufficiently cheap that even if they are not notably skillful, a private company has a hard time asking much of a price for their product. The way forecasting companies keep costs down is to not pay their forecasters very much relative to their workload. This tends to produce forecasts that are only marginally better than those of the "free" public forecasters, who generally are overloaded with duties and cannot put much into any particular user's needs.

Personally, I think a specialized forecast is rather difficult, and the forecasters doing such a job should be well-paid and provided with the best forecasting tools money can buy. If a company followed this practice, and didn't have "free" forecasts with which to compete, it could ask a fair price for its service. And the user would have a right to expect a significantly better product than he/she gets from the public forecasters at present. This would tend to foster a scientific competitiveness among forecasting companies, because high quality for a reasonable price ought to be of considerable value relative to mediocrity at a low price. At the moment, the existence of a "free' product makes this competition unlikely. Forecasting ability in terms of the factual "bottom line" would be something to pay well for, because that would be the basis for competition in the marketplace.

 

The Role of Verification

This brings up verification. Forecasts not verified are not much good to anyone, in my opinion. Verification allows forecasters to know, quantitatively and objectively, how well they are doing, and in what ways they can improve their product. In these days of "total quality management" and "quality assurance," what company would sell a product whose quality went unchecked? Today's quality-conscious market wouldn't tolerate that for long. Yet we continue to do pitifully inadequate verification of our forecasts, public and private, committing such sins as relying on single measures to assess the quality of a multidimensional product. And we do virtually nothing systematic to follow-up that verification seeking to find out how to improve our forecasts. This latter task, to which I refer as "loop-closing," is essential and yet remains virtually ignored in the forecasting business. How often do forecasters get to go back and re-evaluate a situation gone bad with an eye to preventing similar occurrences in the future? Perhaps a pale, whitewash-covered imitation of this occurs when weather disasters kill people, but how often is it done for busts of "ordinary" weather forecasts, or agricultural and aviation weather forecast failures? Virtually never.

In my view of this topic, it's absurd to think that a well-run company producing specialized forecasts would not have an aggressive quality management program. These would not be "forecast police" but a team of people whose job it would be to help forecasters generate and maintain a product quality that might well be substantially better than what we are used to today. Public forecasters are simply not able to keep up with all the needs of all the users. Doing the general forecast and dealing with hazardous weather stretches their capabilities. In no way can they pay strict attention to the details needed for special interests and yet their attempts to do so simply stifle the attempts of the private sector to fill that void.

Public forecasters could have a really important role in a revised system wherein specialized forecasts are produced by private sector forecasting companies. In the transition to such a system, public forecasters could fulfill an educational role for their users, leading them to an understanding of how to use decision-making in their commercial ventures. It seems to me that this ought to be an important part of providing weather information to the public; not simply weather data and forecasts, but information about how to get desired results from the available products, and where to seek additional information. If the public sector were to take a real leading role in this transition, it would have enormous value to the income-generating side of the public-private relationship, which could not avoid helping our society as a whole. There is also going to be a continuing educational need within the public, especially if the public forecasts were to be converted to probabilistic formats for most, or all, of their products. Although there certainly would be resistance to this change, I continue to believe that it would serve the public's needs better, in the long run, than the continued deception that we can issue categorical products with virtual certainty. There was (and continues to be) some resistance to PoPs, but now most people accept them as the way in which precipitation is forecast. We need not assume that the public is stupid and beyond education; that invariably is a self-fulfilling prophecy.

 

Hazardous Weather Warnings

So what about the other, arguably most important part of the public forecaster's job: hazardous weather warnings? All of the foregoing applies here, as well. Hazardous weather has the same sorts of uncertainties attached to it as plain old ordinary weather. Sure, if we actually see a severe event happening, then that particular uncertainty has been removed. That's true of ordinary weather as well. So what? There are other uncertainties, of course. Is that event going to continue or will it dissipate shortly? Where will the event be in the next hour, if it lasts? Will the intensity of the event increase or decrease? And so on. Moreover, the "lead time" associated with waiting until the event has been confirmed is not very useful to the public. They want the same thing from hazardous weather warnings they want from the ordinary forecast: categorical perfection. And we have the same problem providing it: we can't.

In fact, many within the public feel that it is someone else's responsibility, other than their own, to ensure their personal safety from weather hazards. Blaming the public-sector forecasters is a convenient way to sidestep the issue of who is responsible for personal safety. I believe the public has a right to expect information about the threat of hazardous weather, but they need to learn how to use this information. In certain parts of Texas, for example, it turns out that specific local officials are formally charged with the responsibility for ensuring public safety in hazardous events. Their discharge of this formal responsibility takes the form of emergency planning in the event of severe weather, hazardous chemical spills, fires, etc. Local citizens volunteer to be weather spotters, others staff the amateur radio communications links in an emergency operations center. The community, in effect, takes a certain amount of the responsibility onto itself.

This is another area where I believe we have done a poor job (exceptions exist, of course!) in connecting with our users. We can and should be helping the communities (a) understand the reality of what we can and can't do, (b) develop mechanisms by which high-priority weather information can be sent without delay between the local community and the public weather services, and (c) promote awareness of weather hazards and educate the public about what is an appropriate response in the event some hazardous weather develops, or is imminent. With regard to (b) it is all too common in the public weather business to believe that the communication should be from the public weather service down to the community, with little or no input from them coming back to disturb us in our sanctum sanctorums. A properly educated and trained user community will want as much weather information as we can send them, and they may be able to offer useful input for our decision-making process, in return. What we need to foster is a sense of partnership with the communities, not a paternalistic view of them as the ignorant recipients of our wisdom. Our information is for their benefit, and they have a right to expect us to provide that information.

As with the ordinary forecast, though, they need to know our limitations. For example, we cannot see from our vantage point that a tornado is definitely about to hit a certain part of an extended community. They can, and the responsibility to deal with that imminent threat is largely theirs; our responsibility is mostly to the communities downstream, who are wondering if the storm is headed their way, and if it is likely to dissipate, strengthen, or continue unchanged over the next 30 min. as it approaches. Public-sector warnings ought to be aimed at the downstream (in space and time) communities, not the ones being hit right now. The local communities should be responsible for the local warnings, not the public sector forecasting offices. Our warning job, done well or done poorly, is over when the event is underway; the communities have to deal with the ongoing events. If the public weather services have any continuing responsibility for such a community, it is to forecast the cessation of the event, and to be concerned about whether or not new hazards will develop. If someone in a community is "struck without warning" it ought to be due to the failures of their community to develop a proper use of the weather information we in the public weather services have provided. The public sector weather services should not be in the business of telling Mr. and Mrs. Jones that they are about to experience a flash flood, or whatever. Nor should the public think that we are!

 

A Summary

In summary, it seems to me that we in the weather community are suffering from a fear that if we tell the public what we are capable of, they will cut off our funding. When people say "Forget that probability crap. Just tell me whether or not it's going to rain!" they are telling us they want something we are unable to provide. If we meekly acquiesce to this, like a lieutenant crisply saluting the general and saying "Can do, sir!" in the face of impossible odds, we are only sowing the seeds of further discontent and enhancing the very common view of our products as almost worthless. It leads to the very questions (about the value of supporting what we do) that we fear the most. Why not give the public some credit and tell them frankly and honestly what we can do, and then rolling up our sleeves and helping them learn how to deal with the uncertainty in our products? How often has honesty been tried? Has the "can do" mentality really reduced our internal fears of being cut out of the budget? Is it right for the public to continue to expect the impossible from us and to demand that we accept the burden of responsibility for their safety and economic well-being? I think it's time to "renegotiate our contract" with the public, aiming to develop a relationship based on an honest assessment of our abilities to provide weather information to our users. No more "can do" promises we can't keep, no more exaggerations of our ability to deliver, no more political haggling with a public fed up with politics! I think the public is ready to listen if we would simply be honest with them.

I have suggested it is quite difficult for us to point to the economic value of what we do. It may turn out that we will get a chance to see just how valuable we really have been, as budget pressures increase as weather services are severely cut back, perhaps even eliminated. How long will it take to recognize the absence of our input? By the time it eventually is decided (if it is so decided!) that public weather services were a good idea after all, it may be difficult to reconstruct the system. Thus, I believe we should be facing the problem right now and attempting to revise our relationship to our users before it is too late.

 

Research Responsibilities

We collectively have a responsibility to the people that pay the bills for our products. Even in the research community, it is becoming increasingly obvious that total irrelevancy is something we cannot afford. If we could continue to do research at little or no cost, then basic, knowledge-for-its-own-sake science could continue to run its course. But there can be no avoidance of the economic pressures that we all confront. I am not advocating a program solely of "applied" meteorological research; but it is not unreasonable, I believe, to expect some useful results from some portion of our research. It is naive and unrealistic to continue to believe that scientists are so "pure" that they are uncaring about market pressure; a simple consideration of how research follows what is "trendy" is a direct reflection of the willingness of scientists to bend their research to fit what is determined to be important by the funding agencies. All that is needed is for the funding agencies to put an emphasis on operationally relevant subject matter and the researchers will queue up to send in their proposals. Scientists tend to resent the implication they are aware of budget pressures in what they choose to study. This is a self-deluding fiction that applies to very few real scientists. I continue to believe in research for its own sake, and there always should be mechanisms to fund such research, but I am confident if we don't show meaningful results in topics that matter to our society, all such support will cease.

Is it such a compromise of our scientific principles to do relevant work? My conception of meteorology is that a lot of relevant issues also have roots in basic questions of our science. It does not seem so awful to me that our society should ask us to produce something that is of value. Our forecasts are, after all, the mechanism by which our science contributes to our society's well-being. Saving lives is clearly of value, even when we can do little or nothing to reduce many of the property losses due to weather. And there always is the value of the "ordinary" forecast to businesses of all sorts; I don't feel it cheapens our principles to be worried about such mundane topics. If we can redefine our relationship to our users to reflect the reality of our capabilities, I believe everyone will be more satisfied with the situation than they are now.