Monday, 13 April 2015

Reference Class Forecasting or Comparison Class Forecasting

Reference class forecasting, or comparison class forecasting, is the method of predicting the future, through looking at similar past situations and their outcomes. A technique where forecasts of an initiative’s (project or programme) duration, costs and benefits are derived from what actually occurred in a reference class of similar programmes or projects. Alternatively, estimates can be built up in the traditional manner and then adjusted by set percentages based on past performance – this is the approach used in the UK central government where a standard set of optimism bias (described in the section below) adjustments are included in the HMT Green Book.

Reference class forecasting predicts the outcome of a planned action based on actual outcomes in a reference class of similar actions to that being forecast. The theories behind reference class forecasting were developed by Daniel Kahneman and Amos Tversky. This theoretical work helped Kahneman win the Nobel Prize in Economics.

Kahneman and Tversky found that human judgment is generally optimistic due to overconfidence and insufficient consideration of distributional information about outcomes. Therefore, people tend to underestimate the costs, completion times, and risks of planned actions, whereas they tend to overestimate the benefits of those same actions. Such error is caused by actors taking an "inside view," where focus is on the constituents of the specific planned action instead of on the actual outcomes of similar ventures that have already been completed.

Kahneman and Tversky concluded that disregard of distributional information, that is, risk, is perhaps the major source of error in forecasting. On that basis they recommended that forecasters "should therefore make every effort to frame the forecasting problem so as to facilitate utilizing all the distributional information that is available". Using distributional information from previous ventures similar to the one being forecast is called taking an "outside view". Reference class forecasting is a method for taking an outside view on planned actions.

Reference class forecasting for a specific project involves the following three steps:
  • Identify a reference class of past, similar projects.
  • Establish a probability distribution for the selected reference class for the parameter that is being forecast.
  • Compare the specific project with the reference class distribution, in order to establish the most likely outcome for the specific project.
Optimism Bias

There is a demonstrated, systematic, tendency for project appraisers to be overly optimistic. This is a worldwide phenomenon that affects both the private and public sectors.11 Many project parameters are affected by optimism – appraisers tend to overstate benefits, and understate timings and costs, both capital and operational. To redress this tendency, appraisers should make explicit adjustments for this bias. These will take the form of increasing estimates of the costs and decreasing, and delaying the receipt of, estimated benefits. Sensitivity analysis should be used to test assumptions about operating costs and expected benefits. Adjustments should be empirically based, (e.g. using data from past projects or similar projects elsewhere), and adjusted for the unique characteristics of the project in hand. Cross-departmental guidance for generic project categories is available, and should be used in the absence of more specific evidence. But if departments or agencies have a more robust evidence base for cost overruns and other instances of bias, this evidence should be used in preference. When such information is not available, departments are encouraged to collect data to inform their estimates of optimism, and in the meantime use the available data that best fits the case in hand. Adjusting for optimism should provide a better estimate, earlier on, of key project parameters. Enforcing these adjustments for optimism bias is designed to complement and encourage, rather than replace, existing good practice, in terms of calculating project specific risk adjustments. They are also designed to encourage more accurate costing. Accordingly, adjustments for optimism may be reduced as more reliable estimates of relevant costs are built up, and project specific risk work is undertaken. Both cost estimates and adjustments for optimism should be independently reviewed before decisions are taken.

Source and full details - HMT Green Book --

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/220541/green_book_complete.pdf

____________________________________________________
Author - Vijayakumar Reddy, CTO & Lead Trainer, A2A IMTCS Pvt. LTD.

© Copyright 2015 A2A - IMTCS. All rights reserved. www.iimtcs.in
The Swirl logo is a trade mark of AXELOS Limited.
ITIL® is a Registered Trade Mark of AXELOS Limited.
PRINCE2® is a Registered Trade Mark of AXELOS Limited.
MSP® is a Registered Trade Mark of AXELOS Limited.
MoP® is a Registered Trade Mark of AXELOS Limited.
Copyright © AXELOS Limited 2007, 2009 & 2011. Reproduced under license from AXELOS
_____________________________________________________________________