What Monte Carlo Cannot Do: Introduction to Imprecise Probabilities

A tutorial workshop
to be held in conjunction with the

Society for Risk Analysis Annual Meeting

9:00 - 5:30 pm
Sunday, 4 December 2005

Lake Buena Vista (formerly Wyndham) Palace Resort and Spa
Lake Buena Vista, Florida

This tutorial introduces the notions of interval-valued probability and imprecisely specified probability distributions and reviews their uses in risk analysis. It will address the approaches of interval probabilities, probability bounds analysis, Dempster-Shafer theory, robust Bayes methods, and the theory of imprecise probabilities. It will also illustrate how sparse, incomplete or imprecise data can be used to fashion inputs for these models.

Synopsis
Overview of topics
Presenters
Registration
Venue
More information
Related links

 

Synopsis

This full-day tutorial introduces the notions of interval-valued probabilities and imprecisely specified probability distributions and their uses in risk analysis.  It reviews five practical and quantitative approaches based on these elementary notions.  The simplest approach uses the idea of interval probability, in which the probability of an event can be specified as an interval of possible values rather than only as a precise one.  This idea, dating from George Boole, provides a convenient way to assess the reliability of fault-tree risk analyses.  This idea is generalized by probability bounds analysis, which propagates constraints on a distribution function through mathematical operations, and Dempster-Shafer theory which recognizes that uncertainty attending any real-world measurement may not allow an analyst to distinguish between events in empirical evidence.  These approaches are related to robust Bayes (aka Bayesian sensitivity) methods, in which an analyst can relax the requirement that the prior distribution and likelihood function must be precisely specified.  The most general approach comes from the theory of imprecise probabilities in which uncertainty is represented by closed, convex sets of probability distributions.

These five approaches redress, or comprehensively solve, several major deficiencies of Monte Carlo simulations and of standard probability theory in risk assessments.  For instance, it is almost always difficult, if not impossible, to completely characterize dependencies among variables in a risk analysis.  As a result, in the practical situations where empirical data are limiting, these difficulties can result in assessments that are arbitrarily over-specified and therefore misleading.  More fundamentally, it can be argued that probability theory has an inadequate model of ignorance because it uses equiprobability as a model for incertitude and thus cannot distinguish uniform risk from pure lack of knowledge.  In most practical risk assessments, some uncertainty is epistemic rather than aleatory, that is, it is incertitude rather than variability.  For example, uncertainty about the shape of a probability distribution and most other instances of model uncertainty are typically epistemic.  Treating incertitude as though it were variability is even worse than overspecification because it confounds epistemic and aleatory uncertainty and leads to risk conclusions that are simply wrong.  Approaches based on imprecise probabilities allow an analyst to keep these kinds of uncertainty separate and treat them differently as necessary to maintain the interpretation of risk as the frequency of adverse outcomes.

The interval and imprecise probability methods also make backcalculations possible and practicable.  Backcalculation is required to compute cleanup goals, remediation targets and performance standards from available knowledge and constraints about uncertain variables.  The needed calculations are notoriously difficult with standard probabilistic methods and cannot be done at all with straightforward Monte Carlo simulation.

Although the five approaches arose from distinct scholarly traditions and have many important differences, the tutorial emphasizes that they share a commonality of purpose and employ many of the same ideas and methods.  They can be viewed as complementary, and they constitute a single perspective on risk analysis that is sharply different from both traditional worst-case and standard probabilistic approaches.  Each approach is illustrated with a numerical case study and summarized by a checklist of reasons to use, and not to use, the approach.

The workshop will also review how analysis of data sets can benefit from adopting imprecise probability methods. The task, sometimes called "knowledge discovery", of inferring models that make knowledge about a subject explicit typically assumes that empirical data are the only source of information. Thus, learning from data is presumed to start under a condition of prior ignorance. The available data are often imprecise due to measurement uncertainty or incomplete, such as when values are missing in the data set. This constitutes another form of ignorance that is about the data themselves. Modeling these forms of ignorance carefully is a central feature needed to make models and their applications reliable. The workshop will show how imprecise probability allows one to reliably handle incomplete data in a way that significantly departs from established approaches. This depends on the ability to work with weak assumptions about the data.

The presentation style will be casual and interactive.  Participants will receive a CD of the illustrations used during the tutorial.




Overview of topics

What’s missing from Monte Carlo?

Correlations are special cases of dependencies
Probability theory has an inadequate model of ignorance
Model uncertainty is epistemic rather than aleatory in nature
Backcalculation cannot be done with Monte Carlo methods

Robust Bayes and Bayesian sensitivity analysis

Bayes’ rule and the joy of conjugate pairs
Dogma of Ideal Precision
Classes of priors and classes of likelihoods
Robustness and escaping subjectivity
Imprecise Dirichlet model
Case study: medical diagnosis
Why and why not use robust Bayes

Dempster-Shafer theory

Indistinguishability in evidence
Belief and plausibility
Convolution via the Cartesian product
Case study: reliability of dike construction
Why and why not use Dempster-Shafer theory

Interval probability

Conjunction and disjunction (ANDs and ORs)
Fréchet case (no assumption about dependence)
Mathematical programming solution
Case study: fault-tree for a safety subsystem
Why and why not use interval probability

Probability bounds analysis

Marrying interval analysis and probability theory
Fréchet case in convolutions
Backcalculation
Case study: exposure of birds to agricultural insecticide
Why and why not use probability bounds analysis

Handling imperfect data

Sources of incertitude
Statistics for interval data
Missing-at-random (MAR) data
Censored data
Tradeoff between precision and sample size
Contrast with standard method

Imprecise probabilities

Comparative probabilities
Closed convex sets of probability distributions
Multifurcation of the concept of independence
Why and why not use imprecise probabilities



Presenters

Scott Ferson is Senior Scientist at Applied Biomathematics. His research focuses on developing reliable mathematical and statistical tools for risk assessments and on methods for uncertainty analysis when empirical information is very sparse. Ferson holds a Ph.D. from the State University of New York at Stony Brook. He is author of RAMAS Risk Calc Software 4.0: Risk Assessment with Uncertain Numbers (Lewis Publishers). He has written over 75 other scholarly publications, including four other books and several software packages, in environmental risk analysis and uncertainty propagation. His research has addressed quality assurance for Monte Carlo assessments, exact methods for detecting clusters in small data sets, backcalculation methods for use in remediation planning, and distribution-free methods of risk analysis appropriate for use in information-poor situations.

W. Troy Tucker is a human ecologist and anthropologist. He received his doctorate from the University of New Mexico in 1998. He has more than ten years experience in mathematical modeling and in collecting and analyzing data. Past projects include a quantitative statistical study of the demography of New Mexican men and smaller quantitative demographic studies among a hunter-gatherer population in Venezuela, an agricultural village in Tanzania, and swidden agriculturalists in Madagascar. At Applied Biomathematics, his research has focused on risk perception in cooperation with Pfizer, developing case studies and methods to test probabilistic deconvolution and probability bounds under grants from the National Institutes of Health, and developing and testing methods for the incorporation of human demography into ecological risk analyses under a grant from the National Science Foundation. He also has several years experience conducting probabilistic human health and ecological risk assessments for a Superfund site.


Registration

The registration fee is $175 before 4 November, or $235 on site. You do not need to register for the Annual Meeting to attend the workshop. Registration will be handled by

Secretariat sra@burkinc.com
Society for Risk Analysis www.sra.org
1313 Dolley Madison Boulevard, Suite 402
McLean, Virginia 22101 USA
1-703-790-1745, fax 1-703-790-2672



Venue

The event will be held 9:00am - 5:30pm on Sunday, 4 December 2005, at

Lake Buena Vista (formerly Wyndham) Palace Resort & Spa in the Walt Disney World Resort
1900 Buena Vista Drive
P.O. Box 22206 Lake Buena Vista, Florida 32830-2206
Phone: 1-407-827-3333
1-800-996-3426
Fax: 1-407-827-3472
http://www.wyndhamevents.com/palace/05sfra.htm

The room for the workshop has not yet been determined; check with the hotel concierge.

To reserve a room at the hotel, call 1-407-827-3333 or 1-800-996-3426, or fax 1-407-827-3472, or visit the website http://www.wyndhamevents.com/palace/05sfra.htm. SRA has reserved a block of rooms at a special meeting rate of $168.14 per night for single or double occupancy. (This represents a base rate of $139 plus $29.14 per night tax and resort fee.) Once this block of rooms is sold out, rooms may be available at the prevailing rate. Reserve your room early. The cut off for this rate is 4 November 2005 or until the SRA room block is sold out. Be sure to mention the Society for Risk Anlysis to receive the SRA meeting rate. This rate is available three days prior and after the meeting dates of 4-7 December 2005, subject to availability. Cancellations must be made at least 5 days in advance. There is no charge for local or long distance calls up to 30 minutes. After that, the charge is 10 cents per minute for local calls and 20 cents per minute per minute for long distance in the continental United States. There is complimentary self parking at the hotel; valet rates are $11 per day and include in/out service. Internet access is available for $9.95 per calendar day.




More information

More information can be obtained from Scott Ferson scott@ramas.com, telephone 1-631-751-4350, fax 1-631-751–3435.




Related links

The Imprecise Probabilities Project http://www.sipta.org/

Summer School on Imprecise Probabilities http://www.idsia.ch/~zaffalon/events/school2004/topics.htm

Society for Risk Analysis Annual Meeting http://www.sra.org/events_2005_meeting.php

Society for Risk Analysis www.sra.org