Module 5: Risk and Uncertainty in Cost-Benefit Analysis

1. Risk versus Uncertainty
2. Expected Value Analysis
3. Sensitivity Analysis
4. Options Analysis
5. Conclusion

A famous quote attributed to Pliny the Elder is that "the only certainty is uncertainty." This statement clearly describes cost-benefit analysis, where lack of information about the consequences of actions and the benefits and costs of these consequences often confounds the analysis. This essay has two primary purposes. The first purpose is to give the decision maker and the analyst conducting cost-benefit analysis a better understanding of the nature of risk and uncertainty and the ways they can manifest themselves in cost-benefit analysis. The second purpose is to outline methods for incorporating uncertainty into cost-benefit analysis and into decision making processes that use cost-benefit analyses as inputs.

1. Risk versus Uncertainty

Risk and uncertainty are often used interchangeably in casual discussion, but they have very different technical meanings. Risk is defined as the variation in potential outcomes to which an associated probability can be assigned. In statistical terms, the distribution of the variable is known, but not the value from the distribution which will be realized. In poker terms, you know the probability of being dealt the ace of spades, but you do not know if the next card that will be dealt is the ace of spades. In sharp contrast, uncertainty is a lack of knowledge concerning the distribution of the variable. Not only do you not know the next card to be dealt, but you may not know how many cards are in the deck, or how many of those cards are aces of spades.

In life, as in cards, risk is less of a problem that uncertainty. Because risk is associated with probability, risk can be accommodated through the purchase of insurance or hedging. For example, when a Black Jack dealer is showing an ace, the other players may purchase insurance to protect against the dealer having twenty-one. Given the number of cards and the distribution of those cards, the likelihood the dealer has twenty-one could be calculated. Once calculated, the players would know whether or not it was prudent to purchase insurance. Similarly, you do not know if you will be in an automobile accident next year. But because the probability of being in an accident is known, you can buy insurance to protect against that unfortunate outcome.

Uncertainty on the other hand, is the lack of knowledge concerning the probability distribution of future events. This implies that insurance is unavailable to protect against negative outcomes. Therefore, it is essential that the analyst must incorporate uncertainty into the cost-benefit analysis and that the decision maker incorporate uncertainty into the decision process. A lack of knowledge does not preclude making assumptions concerning potential outcomes that should be taken into consideration. Imagine, for example, that a municipality is considering building a garbage to energy incineration plant. An analyst examining the viability of the project could hypothesize that the future would look very similar to the present. That is, population would remain stable, incomes in the municipality would remain constant relative to prices, the volume of garbage would remain relatively stable, the price of energy would not change, and the town's preferences for environmental quality would not change. However, it should be expected that at least one, if not all, of these characteristics will change over the life of the incineration plant. Nevertheless, uncertainty is an attribute of virtually all decision process.

How do we treat uncertainty about key variables in cost benefit analysis? There are three major methods for doing this:

• expected value analysis
• sensitivity analysis
• evaluating "options"

2. Expected Value Analysis

Expected value analysis is designed to deal with risk and uncertainty by assigning probability estimates to alternative outcomes and then using these probability estimates to compute an expected outcome. The value of each outcome is multiplied by its probability and the expected value is computed according to the following formula:

For example, if you by a lottery ticket for one dollar with a 1 in 60,000 chance of winning $10,000 and a 1 in 600,000 chance of winning $100,000, the expected value of your decision to purchase the lottery ticket is

In our example of the incineration plant, we only know the current price of electricity, but we can attach probabilities to potential future prices. For example, assume that the current price is $0.08 per kilowatt hour, and we think that there is a fifty percent chance that it stays the same in 20 years, a 25 percent chance that it falls to $0.06 and a 25 percent change that it increases to $0.20. Then the expected price of energy 20 years in the future is

Of course, a key question here is how to formulate the probability estimates. For variables such as energy prices and population growth, one can look to well developed forecasting models that predict these variables and have standard errors associated with the estimates. However, many times the analyst or decision maker will be confronted with variables for which there are no such forecasting models, such as the growth in the per capita volume of garbage. In this case, the analysts (or experts that the analyst recruits) will need to make subjective probability estimates. The analyst or the expert would take into account various factors such as the changing age distribution of the population, predicted changes in income, and how they feel attitudes will change towards the environment and towards convenience products and make forecasts or future garbage streams and subjectively attach probability estimates to those forecasts.

Although expected value analysis incorporates aspects of the probabilistic nature of important variables, it does not usually incorporate all of the information that is known about the uncertainty of the variable. Thus, although the development of subjective probabilities is one way of treating uncertainty, it is not a complete treatment. One reason for this is though probabilities are estimated, the method does not seek to evaluate the quality of the information underlying the probability estimates. The importance of this can be better understood by looking at the type of probability example normally presented in statistics textbooks. Imagine that there is a box with 100 balls in it, some red and some blue. In the absence of any additional information, one would formulate a subjective probability estimate that the probability of drawing a red ball from the box is 50 percent. Now imagine that you are told that of the 100 balls, at least 10 are red and at least ten are blue. Your subjective probability estimate remains as a 50 percent chance of drawing a red ball, but you have additional information as well. Now you know that there is no less than a 10 percent chance of drawing a red ball, and no greater than a 90 percent chance. In both cases, the subjective probability estimate is 50 percent, but there is more information in the second case, and it is essential that this information be incorporated into the decision making process.

Unfortunately, this type of information is not incorporated into the typical expected value analysis, so that no distinction is made between a 50 percent probability which represents the best guess from a spread of 0 to 100 percent, and a 50 percent probability which represents the best guess between 45 and 55 percent. Either subjective upper and lower probability estimates, confidence intervals, or sensitivity analysis should be employed to better characterize the risk and uncertainty associated with proposed projects.

One reason that these observations are important is that whereas expected value analysis indicates a "most likely value" it incorporates the assumption that such a value results from a series of repeat events. Thus, if the frequency of aces in a deck of cards is 4, the probability of drawing an ace is 1 in 13, but for 48 other draws, a card other than an ace will be drawn. The expected value analysis abstracts from this process, in reality, there may be only one opportunity to draw. In other words, the ability to calculate an expected value is different than possessing the ability to hedge.

Another attribute of expected value analysis is that it assumes that the decision maker places the same weights on gains as on losses whereas, in fact, the weights may be different. This is simply to say that an individual may suffer harm from a bet in which $1,000 is lost that is greater than the well being he feels from winning a $1,000 bet. The analysis must also be careful to specify the source of harm or well being properly. Whereas an individual may place an equal value on an hour of her time saved or wasted, she will not be indifferent between arriving for a plane 15 minutes early or 15 minutes late.

Finally, individuals may evaluate risky situations differently than certain ones. An individual who declines a "fair" wager, for example, is said to be risk averse. In general, individuals tend to be risk averse. Nevertheless, it can be argued that society as a whole should be risk neutral in evaluating uncertain events.

3. Sensitivity Analysis

Sensitivity analysis is a method for analyzing uncertainty by changing input variables and observing the sensitivity of the result. For example, if a positive present value is calculated for a range of discount rates, the analyst can conclude that uncertainty over which discount rate to use does not factor heavily in the analysis. The method can be employed either on a variable-by- variable analysis basis or by changing groups of variables at once using scenario analysis. These are closely related techniques that offer several advantages over other methods for examining the affects of uncertainty. This section explains how both sensitivity and scenario analysis can be employed in cost-benefit analysis to provide decision makers with improved information. The discussion covers the methods of calculation, and advantages and disadvantages of each technique. Lastly, alternative methods for incorporating uncertainty are mentioned.

Variable-by-Variable Analysis. Sensitivity analysis is a simple and effective means for analyzing uncertainty, that isolates the affect of a change in one variable on the cost-benefit ratio. This method is also referred to as the variable-by-variable approach. There are four steps to employing the variable-by-variable approach.

1. List all of the important factors that affect the cost-benefit flows.
   
2. For each factor define a range of possible values. The range usually consists of three to five values. These can be based on any relative measure. For example, estimates for each factor could be prepared under "optimistic', "most likely", or "pessimistic" future states of the world. In practice, these values are usually based on past experience with similar projects or expert opinion. Occasionally, the range is even expressed as one or two standard deviations from a mean or expected value.
   
3. Calculate cost-benefit ratios or net present values for each value of each factor holding all other factors at their expected or most likely values. This means that if there are three factors and three estimates for each factor seven different benefit/cost ratios will be calculated.
   
4. The resulting cost-benefit ratios or NPV's should be examined to determine the degree of overall variation and which factor or factors is/are most responsible for variation in the estimates.
   

This process can be illustrated by examining the decision of whether or not to build a new baseball stadium, where there is uncertainty about the costs of construction, as well as the degree of fan enthusiasm for attending games in the new stadium. The table below illustrates steps one and two of the method employing this example.

The first column lists the factors believed to be most important in the determination of cost-benefit flows. The numerical values are the estimates of the cost-benefit flows generated under various assumptions. Once values for each factor have been determined, the next step is to calculate a net present value or cost-benefit ratio using the most likely values for each. Then additional net present values or cost-benefit ratios are calculated by allowing one factor to vary while the others are held constant at their most likely values. If we assume that the values in the table are already expressed in present value terms, we can calculate a series of cost-benefit ratios. These are shown in the following table.

The results show how sensitive the cost-benefit ratios are to changes in individual factors. For example, little variation is caused by fluctuations in the revenues from concession sales. On the other hand, the cost-benefit ratios are very sensitive to changes in either construction costs or ticket sales. However, only the combination of pessimistic ticket sales and most likely estimates for concessions and construction costs fails to generate a positive cost-benefit ratio.

Scenario Analysis. Scenario analysis is based on the assumption that factors affecting cost-benefit flows do not operate independently of one another as is assumed in the variable-by-variable approach. For example, it is unlikely that ticket sales and concessions are independent factors. Greater ticket sales should imply greater concession sales. Of course, fewer ticket sales imply fewer concession sales as well. This realization allows us to combine levels of various factors in consistent combinations. If we reexamine the baseball stadium example using scenario analysis, the problem can be both simplified and offer better information on which to base a decision.

Assume we have divided the potential future states of the world into best, worst and most likely scenarios. The best case scenario is based on the lowest estimate for construction costs and the most optimistic estimates for ticket and concession sales. The worst case scenario is obviously based upon the most pessimistic estimates for construction costs, ticket sales, and concessions. Again, refer to the table below.

Comparing the results shown in the table above with the results of the variable-by-variable approach the most striking difference is the degree of variation. The lowest cost-benefit ratio calculated using the variable-by-variable approach was 0.88, which is significantly greater than the worst case scenario outcome of 0.3. The greatest cost-benefit ratio calculated using the variable-by-variable approach was 5.2, which is less than half the value calculated given the best case scenario. The scenario approach allows the decision maker to observe under which scenario or group of factors does the proposed project perform best, worst, etc.

Advantages and Disadvantages. Sensitivity analysis has several advantages. First, it is relatively easy to compute the necessary information required for either approach. In fact, the researcher can simply assume a range of values around the most likely case, without undertaking a great deal of work. This is less true for scenario analysis than sensitivity analysis. Second, the process provides more information upon which to base a decision. In particular, it provides a notion of where the impacts of uncertainty are important for the analysis and where they are not. This could cause the analyst to gather additional information. Third, because the process requires a careful examination of the factors most likely to influence the cost-benefit flows, the analysts is better informed as to what the results of the analysis truly represent. Finally, the potential interaction of factors is revealed when scenario analysis is employed.

Several disadvantages are also prevalent. First, the determination of values that correspond to variations in key factors is based upon the best information at the disposal of the analyst. Inevitably, this implies the reliance on ad hoc methods for determining pessimistic, optimistic and most likely estimates. Also, the lack of a systematic method for determining the appropriate combination of factors used to define given scenarios limits the reliability of sensitivity analysis. Finally, while the variable-by-variable approach fails to account for factor interaction, the scenario approach usually only includes a small number of potential scenarios.

4. Options Analysis

Options analysis is a general term that refers to the analysts stepping back from the analysis and asking if the analysis is framed in the only way possible or if there are additional options that could better manage the uncertainty faced by the analysis. In general, two types of options analysis are available, one is sequential decision analysis and the other is the approach of irreversible investment theory.

Sequential Decision Analysis. Sometimes activities that appear to be all or none decisions can, in fact, be subdivided into parts, such that information gained during the early parts of the activity can be used to reduce the uncertainty in the later parts of the activity. Such division can sometimes be very trivial. For example, in years past, no one financed a home other than by using a note with a fixed rate of interest over the life of the note. Whereas, this appeared to benefit the homeowner, because the same interest rate would be used, what in fact occurred was that the lender was bearing the risk that other factors, like inflation, would remain constant over the life of the loan. The lender charged a risk premium for this service. When uncertainty over inflation increased, borrowers became aware that by using variable rate mortgages they could often decrease their costs of borrowing. In exchange for these lower costs, the borrowers engaged in risk sharing with the lenders. What in fact was done was the longer term note was effectively broken into a set of shorter term notes with provision to adjust the interest rate as each note came due.

This same principle can be used in many other circumstances. As an example, a utility company, uncertain about the future growth of its electrical load might purchase smaller generators over time, rather than one large generator. A waste clean up activity might divide the larger task into segments. For example, rather than write a single contract to analyze a waste problem and then clean it up, two contracts, one to analyze the problem and a second, to clean up, based on information from the first task, might be employed. Whereas these are simple examples, the important point is that not all environmental decisions are all or none, like a decision to build a dam. Many can be profitably subdivided with resulting decreases in uncertainty.

Irreversible Investment Analysis. In recent years new developments in the theory of investment have led to some very powerful, yet often very simple, observations that are relevant for environmental decision making. The first is that even though cost-benefit analyses are typically framed as whether or not to undertake a project, the analysis might also seek to answer the question when to undertake it. Often projects are posed as now or never. Sometimes, when waiting is possible uncertainty can be resolved at little or no cost. For example, consider the decision by a commuter to purchase an electric car. A cost-benefit analysis would develop data about the expected life of the electric car, the expected costs of alternative means of commuting, and how satisfactory each mode would be. The electric car, might prove to be the most cost effective of the alternatives. But, in fact, the savings calculated by the analysis would eventually be available six months or twelve months hence. Furthermore, over this time period, the commuter could observe the experiences of others with the electric car and the alternatives. The insight of irreversible investment theory is that if uncertain events prove unfavorable, the value of the investment may be totally lost, whereas the cost of waiting may be only the savings given up until the decision is finally made.

This same type of reasoning can be applied to environmental decisions that are irreversible, in the sense that they require the sacrifice of some irreplaceable environmental asset. Hence, if science is uncertain about the role of a particular element of a larger ecosystem, with the potential for high costs if uncertainty resolves unfavorably, there can be significant value to waiting until uncertainty is resolved.

5. Conclusion

All decisions, save the most trivial, are decisions under uncertainty. Hence, the decision maker will never have the assurance, nor the peace of mind, of knowing for sure what the entire consequences of an action may be. Nonetheless, there are ways to hedge against risk and to reduce uncertainty, and we have reviewed several here. The reader should not assume we have exhausted the possibilities. The decision analysis literature is replete with tools and techniques to manage the information content of an analysis. Nevertheless, by applying the basic cost-benefit framework with care, a great deal of uncertainty can be reduced and the probability of an optimal decision increased.

Cost-Benefit Analysis and Environmental Decision Making:
An Overview

Module 1: Increasing Pressures to Use Cost-Benefit Analysis

Module 2: Methods for Determination of Value from Capital Projects

Module 3: Comparing Projects with Different Economic Lives

Module 4: The Choice of Discount Rate

Module 5: Risk and Uncertainty in Cost-Benefit Analysis

Module 6: Marginal Damage Functions

Module 7: Measuring Benefits and Costs.