The Weather and Climate Impact Assessment Science Initiative
    Home Current Projects Uncertainty in Model Simulations
  About the Program
Overview
People
Contact Information
Program Documents
2004 Review Documents
  Research
Current Projects
Publications
Presentations
  Links of Interest
Supporting Institutions
Other NCAR Initiatives
  For Program Staff
Upcoming Events
Mailing List
Website Statistics Initiative Staff Only
  Search This Website

...more search features  
Uncertainty in Model Simulations
 

Project Abstract

A major function of the Intergovernmental Panel on Climate Change (IPCC) is to "assess the state of our understanding and to judge the confidence with which we can make projections of climate change and its impacts." As stated by Moss and Schneider (2000) achieving consistency in and indeed, even understanding how to assign probabilities to outcomes or processes that encompass various types of uncertainties is very difficult even in the research realm. Communicating uncertainty to decision-makers in a form that makes it useful for policy-making is yet another difficult challenge. Uncertainty itself has various meanings and levels—it can refer to lack of knowledge, lack of certainty, disagreement among experts, or the fundamental nature of the scientific process which experiences uncertainties as part and parcel of the experimental method.

Uncertainties in climate model projections have become of interest in recent years because such a wide range of future projections have emerged from a combination of various socio-economic scenarios and different climate models. Decision-makers and scientists alike have expressed a desire to have probabilities assigned to each scenario so that there is a better sense of whether certain scenarios are more likely than others (Schneider 2001). Difficulty in assigning probabilities to scenarios can stem from "epistemic" or "stochastic" sources of uncertainty (Dessai and Hulme 2003). Epistemic sources of uncertainty are those that can be reduced by further study of the system, improving our state of knowledge, etc. Stochastic sources of uncertainty are those that are considered "unknowable"—items such as variability in the system, the chaotic nature of the climate system, and the indeterminancy of human systems. Further, probabilities of various scenarios occurring will likely even change as soon as a prediction is made, because society begins to react and therefore change the outcome in ways the prediction did not incorporate (Sarewitz et al. 2003). Given the intense need of decision-makers for some sort way to evaluate scientific predictions on climate change, Schneider and Dessai and Hulme emphasize the need for researchers to be explicit and transparent about the assumptions used to represent uncertainties, as probabilities or otherwise.

There is therefore a strong need for studying aspects of the uncertainty of climate projections, in order for scientists to be explicit about where estimates of uncertainties come from. This topic has been under-emphasized in the past. The major goals of this project are to develop new techniques for quantifying uncertainty in climate model projections and to apply these techniques to recent transient runs of atmosphere-ocean general circulation models (AOGCMs). Recent emphasis is given to quantifying regional uncertainty. Regional projections of impacts are most needed by decision-makers, and yet are not easily extracted from global climate model simulations. Results can sometimes even be contradictory at the regional scale, with either wetter or drier conditions predicted depending on the model used for the simulation (Dessai and Hulme).

 
Sub-Projects
Low Frequency Variability
Future Climate Change at Regional Scales
From Large-Scale Regional Probability of Climate Change to Watershed-Scale Projections
Extending the Use of Climate Simulations
Climate Modeling for Everyone!
 

Low Frequency Variability

To quantify the uncertainty surrounding the origin of low frequency variability in the observations of globally averaged surface air temperature in 20th century climate, analyses have been undertaken to quantify the influences of various anthropogenic and natural (volcanic and solar) forcings over the 20th century in the NCAR Parallel Climate Model (PCM). Early century warming is shown to be mainly due to solar forcing, with enhanced tropical convection contributing to amplifying the solar forcing. Late century warming is attributed mainly to increases of anthropogenic greenhouse gases (Meehl, et al).

 
Click on image to enlarge
(Click on image to enlarge)
This figure shows the time evolution of globally averaged surface air temperature from multiple ensemble simulations of 20th century climate from the PCM compared to observations. The simulations start in the late 19th century, and continue to the year 2000. The temperature scale at left is in degrees Centigrade, and temperature anomalies are calculated relative to a reference period averaged from 1890 to 1919. The black line shows the observed data, or the actual, recorded globally averaged surface air temperatures from the past century. The blue and red lines are the average of four simulations each from the computer model. The pink and light blue shaded areas depict the range of the four simulations for each experiment, giving an idea of the uncertainty of a given realization of 20th century climate from the climate model. The blue line shows the average from the four member ensemble of the simulated time evolution of globally average surface air temperature when only "natural" influences (solar variability and volcanic eruptions) are included in the model. Therefore, the blue line represents what the model says global average temperatures would have been if there had been no human influences. The red line shows the average of the four member ensemble experiment when natural forcings AND anthropogenic influences (greenhouse gases including carbon dioxide, sulfate aerosols from air pollution, and ozone changes) are included in the model.

Future Climate Change at Regional Scales

We have developed a Bayesian statistical model which compares multiple AOGCM's to historical data and each other, to determine a statistical distribution of a region's future climate (in this case, seasonal absolute changes in temperature and relative changes in precipitation). The regions are subcontinental in scale, and are shown in the map below.

In the Bayesian framework, we first hypothesize prior distributions for the quantities whose uncertainty we want to characterize, e.g. regional temperature, under current and future climate conditions, that we consider as random parameters; we then formulate the likelihood of the data (e.g. observed and modeled temperatures), conditional on the value of the parameters; the two components are combined through Bayes theorem, and the result are, for example, posterior distributions of temperature change.

The statistical assumptions are formulated so that two criteria of climate model evaluation --- bias and convergence --- are going to influence the final result, in the way the members of the multi-model ensemble are "weighted" in the posterior distribution of climate change. Bias is a measure of how well a model reproduces present day climate. Convergence is a measure of how well its future projection agrees with the other models. The regional nature of the analysis will cause the different climate models to have different relative weight in different regions (and seasons), because the performance of a specific model may vary from region to region and seasonally.

We make available detailed Probability Distribution Functions (PDFs) and Cumulative Distribution Functions (CDFs) representations of temperature and precipitation changes for all regions. Click here. More results from this model, including data for custom regions, are available at rcpm.ucar.edu.

 
 
Click on image to enlarge

Figure Introduction to CDFs: This figure presents two cumulative distribution functions --- in this case for Northern Europe, winter-time temperature changes under the A2 and B2 SRES scenarios --- and illustrates how to derive specific information from them. For each probability level, identified along the y-axis, one may derive the corresponding value of temperature change (in degrees Celsius), along the x-axis, by drawing an horizontal line at the chosen probability level, meeting the CDF at the intersection point, and projectin a vertical line down to the x-axis from that intersection. In the figure we show an example for the 0.5 probability level, identifying the values 3.43 and 4.67 degrees C respectively for the CDF associated with B2 temperature changes, and A2 temperature changes. THerefore one can claim that there is 50% chance that the temperature change under scenario B2 will be 3.43 or higher. And the same chance that under A2 the temperature change will be 4.67 or higher. One may be interested in the 95% probability intervals of the two distributions, that can be derived by identifying the values along the x-axis corresponding to the curves' y-coordinates of, respectively 0.025 and 0.975. This would in practice amount to drawing two horizontal lines at these two probability levels, intersecting each curve, and projecting two vertical lines from those points of intersection down to the x-axis. If one did so, the two points identified for the dashed CDF would be 2.4 degrees C (corresponding to probability level 0.025) and 4.6 degrees C (corresponding to probability level 0.975), meaning that the temperature change in Northern Europe, under scenario B2, in winter will be between those two values with 95% probability. Similarly, under scenario A2 the temperature change will be between 3.3 and 6 degrees C.

 
 
Click on image to enlarge
(Click on image to access PDF file)
Figure 1: The cumulative distribution functions of percent precipitation change, with respect to current mean precipitation, for six regions (SEA=South East Asia, CAS=Central Asia, WAF=Western Africa, ENA=East North America, NEU=Northern Europe and ALA=Alaska) in Boreal winter (DJF). Top panel: the six CDFs are directly compared. Steeper curves correspond to narrower probabilities densities (with smaller variance, indicating a prediction concentrated on a relatively narrower range of values). Left-most curves assign larger probability to small values of percent change, compared to curves to the right. The curve for Alaska (ALA) has a strikingly large variance, extending over a range of values that is too wide to be directly comparable to the range of values for the remaining five curves. So we plot it separately in the Bottom panel, where the range of the first panel is indicated by the dashed vertical lines. While the five regions in the top panel show a predicted change for the most part positive (between 0 and 20/25% increase in precipitation) the probability distribution for Alaska extends over a wide range of both positive and negative values, indicating high uncertainty in the prediction. The wide range of the predictive distribution is a consequence of the lack of accuracy of the AOGCM simulations of current climate in the Alaska region. The AOGCMs show biases up to 340% of the present day observed precipitation amount. So our statistical model widens the range of their future projections as a result.
 
Click on image to enlarge
(Click on image to access PDF file)
Figure 2: The six probability densities of precipitation change, corresponding to the six cdfs in Figure 1 (above) are shown here in separate plots, with a red line and triangle at the bottom of each plot indicating mean and 95% probability interval. The ranges are similar for all regions but Alaska. By this representation we can assess more directly the shape of the distribution, in some cases slightly asymmetric, in the case of Alaska extremely diffuse (notice the different range of values on the x-axis). However, all in all, the shapes are regular, unimodal and close to simmetric. These curves may be compared to the corresponding curves for temperature change in Figure 4 (below).
 
Click on image to enlarge
(Click on image to access PDF file)
Figure 3: Same as Figure 1 (above), but for Temperature change. Here the ranges of the probability distributions are more directly comparable, but Alaska still shows a significantly different behavior, concentrating the probability of temperature change over values significantly greater than the other regions'. South East Asia (SEA) is to the left of all the curves, with predicted changes of lower values, comparedto the other five regions. The steepness of the curves for East North America (ENA) and ALA is greater compared to the other curves, indicating less uncertainty in the prediction.
 
Click on image to enlarge
(Click on image to access PDF file)
Figure 4: Same as Figure 2 (above), but for Temperature change. Differently from the densities in Figure 2, these curves show features of multimodality, indicating cases where different subsets of models predict changes in different value ranges, and the different predictions cannot be reconciled, either by discarding models the disagree based on poor performance in the current climate reproductions, or by a large majority concentrating on one interval rather than the others. We do not necessarily associate a physical meaning to the shape of the curves, rather, they signal regions where the prediction is problematic and the various AOGCMs are in little agreement.
 
Click on image to enlarge
(Click on image to enlarge)
The figure to the left is related to the Bayesian analysis. It shows 22 boxplots indicating the range of predicted warming for the 22 regions we analyzed: these boxplots are indicating the extent ofthe distributions together with the higher probability regions (delimited by the boxes). All regions warm, the higher latitude more than the others, some with a larger spread (uncertainty) than others.
 
To learn more about the method used, please click here. (PDF file)
For an overview of the project, please click here. (PDF file)
 

From Large-Scale Regional Probability of Climate Change to Watershed-Scale Projections

The Figures below shows the Western North American region from which Bayesian distributions are derived. The single, large gray point identifies these distributions, two for precipitation and two for temperature. Simultaneously, we can derive seasonal, weekly mean precipitation and temperature values as simply the average of all the historical observations in the region (the small dark marks). With the k-nn technique, these weekly means are then ranked according to the magnitude of their anomaly relative to the mean value found in the historical record- e.g. warmer to cooler years or wetter to drier years, for example. The historic record is then resampled, but biased by the ranked list. The biased resampling produces a list-of-dates from which daily weather for any station in the region can be derived. In this way, weather data at much smaller resolutions (e.g the watershed scale as indicated in the lower Figure) can be produced with regional attributes that are defined by the WNA Bayesian distributions. In FY-04 we will: 1) Obtain daily, historical data for the WNA region; 2) Derive regional mean values and rank this data according to their weekly anomalies; 3) Apply the k-nn technique to produce distributions with similar statistical properties as the Bayesian distributions for the WNA region; 4) Produce Sacramento Watershed-scale weather data from this process, which can be used in a regional water resource assessment model.

Western North American

Watershed Scale

Regional (e.g. Western North America) mean temperature and precipitation values will be computed for each week in the historical record.

 

Extending the Use of Climate Simulations

Pattern scaling, if proven accurate at fine regional scales and if used in concert with a measure of uncertainty in its results, is a way of interpolating between AOGCMs runs under different scenarios (Santer et al. 1990). Pattern scaling could also be an inexpensive substitution to the expensive model runs, especially if the runs that are actually performed by the fully coupled AOGCM are chosen "cleverly" by some experimental design criterion. So we are interested in testing how accurate it is, and how the uncertainty in its results can be characterized.

 
Click on image to enlarge
(Click on image to enlarge)
The figure to left shows the signal of temperature change under two different scenarios (first row), and the result of applying pattern scaling to infer the first scenario only on the basis of the second + a measure of global mean warming under the first (second row first panel). The last panel shows the errors resulting from the approximation.
 

The geographic patterns of climate change, defined as the difference between present and future multidecadal averages of temperature and precipitation, have been found to be constant -- at least for somewhat coarse spatial scales -- across different emission scenarios, as long as these are characterized mainly by long-lived, well mixed gases.This finding is at the basis of the idea of pattern-scaling: if all that changes across different scenarios is the amplitude of the signal, not its geographical distribution, we may save computer time by running many different scenarios on simple, low dimensional, uncoupled climate models, fast and cheap but able to be tuned to reproduce globally averaged results as they come out of complex ocean-atmosphere general circulation models (AOGCMs), then apply the resulting global mean temperature change to modulate the spatial patterns derived once and for all from the latter, run under a single baseline scenario of increased anthropogenic emissions.

We have worked on this idea, exploring the range of spatial scales at which pattern-scaling works, and attaching uncertainty to the scaled signals of climate change by modeling the error as a gaussian random field over the AOGCM grid (Tebaldi, Nychka, Mearns. 2004).

 
Click on image to enlarge
(Click on image to enlarge)
The figure to left plays a little game. One of the panel is the field of errors from the earlier exercise, the other 8 are simulation from a statistical model trying to approximate it. The similarities among the panels are a first order proof than we can characterize the uncertainty of the pattern scaling method through a statistical model, thus attaching confidence bounds to the results of pattern scaling.

Climate Modeling for Everyone!

In addition, users can now directly access the models and data used by the IPCC TAR for its projections of future global-mean temperature and sea level rise through the MAGICC/SCENGEN software package (www.cgd.ucar.edu/cas/wigley/magicc/). It also allows users access to a large data base of AOGCM and observational data. Users can produce regional climate scenarios for changes in the mean state and for changes in variability, including probabilistic projections, and quantify uncertainties in different ways. MAGICC and SCENGEN are coupled, user-friendly interactive software suites that allow users to investigate future climate change and its uncertainties at both the global-mean and regional levels. MAGICC carries through calculations at the global-mean level using the same upwelling-diffusion climate model that has been and is employed by the IPCC. SCENGEN uses these results, together with results from a set of coupled Atmosphere/Ocean General Circulation Models (AOGCMs) and a detailed baseline climatology, to produce spatially-detailed information regarding future changes in temperature and precipitation, changes in their variability, and a range of other statistics.

 

Publications

Meehl GA, Washington WM, Ammann C, Arblaster JM, Wigley TML, Tebaldi C., 2004: Combinations of natural and anthropogenic forcings and 20th century climate. J. Climate, (in press).

Nychka N, Tebaldi C. 2003: Comment on "Calculation of Average, Uncertainty Range and Reliability of Regional Climate Changes from AOGCM Simulations via the ''Reliability Ensemble Averaging'' (REA) method" Journal of Climate: Vol. 16, No. 5, pp. 883-884.

Santer B.D., M.F. Wehner, T.M.L. Wigley, R. Sausen, G.A. Meehl, K.E. Taylor, C.M. Ammann, J. Arblaster, W.M. Washington, J.S. Boyle, W. Brueggemann, 2003: Contributions of anthropogenic and natural forcing to recent tropopause height changes. Science, 301, 479-483.

Santer, B.D., Wigley, T.M.L., Meehl, G.A., Wehner, M.F., Mears, C., Schabel, M., Wentz, F.J., Ammann, C., Arblaster, J., Bettge, T., Washington, W.M., Taylor, K.E., Boyle, J.S., Bruggemann, W., and Doutriaux, C., 2003: Influence of satellite data uncertainties on the detection of externally-forced climate change. Science, 300, 1280–1284.

Tebaldi C, Nychka D, Mearns LO., 2004: From global mean responses to regional signals of climate change: simple pattern scaling, its limitations (or lack of) and the uncertainty in its results. In Proceedings of the 18th Conference on Probability and Statistics in the Atmospheric Sciences, AMS Annual Meeting, Seattle, WA.

Tebaldi C, Smith RL, Nychka D, Mearns LO., Quantifying uncertainty in Projections of Regional Climate Change: a Bayesian Approach to the Analysis of Multimodel Ensembles, accepted subject to revision, Journal of Climate.

 

Project PIs, Leads, and Staff

  • Linda Mearns, PI, Project Lead
    Environmental & Societal Impacts Group, NCAR
  • Doug Nychka, PI, Project Lead
    Geophysical Statistics Project, NCAR
  • Dorin Drignei
    Geophysical Statistics Project, NCAR
  • Gerald Meehl
    Climate & Global Dynamics Division, NCAR
  • Claudia Tebaldi
    Environmental & Societal Impacts Group, NCAR
  • Tom Wigley
    Climate & Global Dynamics Division, NCAR
  • David Yates
    Research Applications Program, NCAR

For more information about this project, please contact Linda Mearns at: lindam@ucar.edu, or Doug Nychka at: nychka@ucar.edu

  Initiative Staff Only Denotes Initiative Staff Only ©2007 UCAR   |   Privacy Policy   |   Terms of Use   |   Top of Page  
NCAR Weather and Climate Impact Assessment Science Initiative