|
Using Inferred Probabilities to Measure the Accuracy of Imprecise Forecasts
February 2011
Paul Lehner, The MITRE Corporation
Avra Michelson, The MITRE Corporation
Leonard Adelman, George Mason University
Anna Goodman, The MITRE Corporation
ABSTRACT
Forecasting research is effectively limited to topics where forecasts are expressed quantitatively; which includes forecasts about future quantities (e.g. population growth) or quantitative probabilities about future events (e.g. election outcomes). This is because quantitative forecasts are expressed with sufficient clarity to support retrospective evaluations of the accuracy of those forecasts. Unfortunately, in real world practice most forecasts are expressed with considerable imprecision. This is especially true of event forecasts where verbal expressions such as "likely", "fair chance" and "strong possibility" are routinely used to characterize forecast certainty. This practice makes it difficult to assess forecasting accuracy; which in turn makes it difficult to empirically investigate the methods used to generate most real world forecasts. This paper presents a methodology for measuring the forecast accuracy of products that are imprecise in their expressions of forecast certainty, by measuring the accuracy of quantitative probabilities that are inferred from the imprecise text. This inferred probability method was tested on ten forecast documents. Results indicate that the obtained accuracy profiles are comparable to those obtained from similar forecasts that were directly expressed as quantitative probabilities, supporting the general utility of this method.
Additional Search Keywords
n/a
|
