Efficient Estimation of the Value of Information in Monte Carlo Models

The expected value of information (EVI) is the most powerful measure of sensitivity to uncertainty in a decision model: it measures the potential of information to improve the decision, and hence measures the expected value of outcome. Standard methods for computing EVI use discrete variables and are computationally intractable for models that contain more than a few variables. Monte Carlo simulation provides the basis for more tractable evaluation of large predictive models with continuous and discrete variables, but so far computation of EVI in a Monte Carlo setting also has appeared impractical. We introduce an approximate approach based on pre-posterior analysis for estimating EVI in Monte Carlo models. Our method uses a linear approximation to the value function and multiple linear regression to estimate the linear model from the samples. The approach is efficient and practical for extremely large models. It allows easy estimation of EVI for perfect or partial information on individual variables or on combinations of variables. We illustrate its implementation within Demos (a decision modeling system), and its application to a large model for crisis transportation planning.

Decision Flexibility

The development of new methods and representations for temporal decision-making requires a principled basis for characterizing and measuring the flexibility of decision strategies in the face of uncertainty. Our goal in this paper is to provide a framework - not a theory - for observing how decision policies behave in the face of informational perturbations, to gain clues as to how they might behave in the face of unanticipated, possibly unarticulated uncertainties. To this end, we find it beneficial to distinguish between two types of uncertainty: "Small World" and "Large World" uncertainty. The first type can be resolved by posing an unambiguous question to a "clairvoyant," and is anchored on some well-defined aspect of a decision frame. The second type is more troublesome, yet it is often of greater interest when we address the issue of flexibility; this type of uncertainty can be resolved only by consulting a "psychic." We next observe that one approach to flexibility used in the economics literature is already implicitly accounted for in the Maximum Expected Utility (MEU) principle from decision theory. Though simple, the observation establishes the context for a more illuminating notion of flexibility, what we term flexibility with respect to information revelation. We show how to perform flexibility analysis of a static (i.e., single period) decision problem using a simple example, and we observe that the most flexible alternative thus identified is not necessarily the MEU alternative. We extend our analysis for a dynamic (i.e., multi-period) model, and we demonstrate how to calculate the value of flexibility for decision strategies that allow downstream revision of an upstream commitment decision.

Decision-Analytic Approaches to Operational Decision Making: Application and Observation

Decision analysis (DA) and the rich set of tools developed by researchers in decision making under uncertainty show great potential to penetrate the technological content of the products and services delivered by firms in a variety of industries as well as the business processes used to deliver those products and services to market. In this paper I describe work in progress at Sun Microsystems in the application of decision-analytic methods to Operational Decision Making (ODM) in its World-Wide Operations (WWOPS) Business Management Group. Working with membersof product engineering, marketing, and sales, operations planners from WWOPS have begun to use a decision-analytic framework called SCRAM (Supply Communication/Risk Assessment and Management) to structure and solve problems in product planning, tracking, and transition. Concepts such as information value provide a powerful method of managing huge information sets and thereby enable managers to focus attention on factors that matter most for their business. Finally, our process-oriented introduction of decision-analytic methods to Sun managers has led to a focused effort to develop decision support software based on methods from decision making under uncertainty.