What is the best tool use in expert judgment?
But first let us define expert judgment and I will try to excerpt it from the report and presentation of my classmates in our Project Management class. Expert judgment is an approach for soliciting informed opinions from individuals with particular expertise. This approach is used to obtain a rapid assessment of the state of knowledge about a particular aspect of climate change. It is frequently used in a panel format, aggregating opinions to cover a broad range of issues regarding a topic. Expert judgment is frequently used to produce position papers on issues requiring policy responses and is integral to most other decision-making tools. This approach is most useful either in conjunction with a full research study or when there is insufficient time to undertake a full study. It is important to be aware, however, of the subjective nature of expert judgment and the need to select a representative sample of experts to cover the full spectrum of opinion on an issue. It is easy to apply but it requires knowledge of policy issues and available experts. More training may be required to assemble an expert panel, formulate questionnaires, and interpret and aggregate expert opinions.
The best is the 'Delphi technique' is a commonly used tool to secure expert judgment while initiating a project. 'Peer review' is a project selection tool, 'Expected value' is a method quantitative risk analysis, and 'WBS' is a project-planning tool. The Delphi method was developed at the RAND corporation in the early 1950s as a spin-off of an Air Force-sponsored research project, “Project Delphi”. The original project was designed to anticipate an optimal targeting of U.S. industries by a hypothetical Soviet strategic planner. In the middle 1960s and early 1970s the Delphi method found a wide variety of applications, and by 1974 the number of Delphi studies has exceeded 10,000 [Linstone & Turoff, 1975]. The Delphi method has undergone substantial evolution and diversification. The method was developed by mathematicians and engineers, and enjoyed considerable popularity among research managers, policy analysts, and corporate planners in the late 1960s early 1970s. By the middle of 1970s psychometricians, people trained in conducting controlled experiments with humans, began taking serious look at the Delphi methods and results. According to Cooke [Cooke, 1991], the most significant study in this regard is Sackman’s Delphi Critique (1975). As a result the whole question of evaluating expert opinion and developing methodological guidelines for its use has moved into the foreground. The Delphi exercises seem to have disappeared, and play almost no role in contemporary discussions of expert opinion.
I will list the methodologies under this technique:
- CTN-UPM,
- FEJ-GRS,
- KEEJAM,
- NNC,
- NUREG-1150,
- STUK-VTT,
- SEJ-TUD.
- LCM
- PMDA-PIRT
- RIPBR
- TRIZ-AFD
CTN-UPM [Cojazzi & Fogli, 2000]
This method was developed in 1997 by the Department of nuclear engineering, University of Polytechnics of Madrid, Spain. It was developed and adapted on the basis of the NUREG-1150 method, although there exists a very important difference between them regarding the way to aggregate experts evaluations. The CTN protocol has been developed to get estimates of subjective probabilities for unknown parameters and uncertain events. It consists of nine steps executed sequentially.
FEJ-GRS [Cojazzi & Fogli, 2000]
This method was developed in 1985 by GRS, Germany. The method has been developed to quantify the state of knowledge in elements of a breakdown of the question and to propagate it through this breakdown to arrive at a quantitative uncertainty statement for the answer. The method aggregates the judgements at lower levels and propagates them through the breakdown to arrive at a quantitative expression of the resulting state of knowledge at the model output level.
KEEJAM [Cojazzi & al, 1996], [Cojazzi & al, 1997], [Cojazzi & al, 1998], [Cojazzi & Fogli, 2000]
This method was developed in 1997 at JRC-ISIS in collaboration with the University of Brescia and the University of Bologna, Italy. Knowledge based method : the method employs Knowledge Engineering techniques, and includes explicit modelling of the knowledge and problem solving procedure of the domain expert. The approach provides structured and disciplined support to the knowledge engineer in eliciting the knowledge and reasoning strategies of the experts, building consistent knowledge models, and applying these models to the solution of the expert judgement task.
NNC [Cojazzi & Fogli, 2000]
This method was developed in 1996. It is based on the quality principles and procedures in the NNC Quality Procedures and Engineering Manual, U.K.. NNC is a Quality based method : based on quality assurance methods of the sources of information and of the problem solving processes, this approach is based on individual estimates. It involves a multi-disciplinary team, defined as a set of individuals with different but complementary skills. As there is no rigorous formal elicitation process, the NNC approach may be called informal expert judgement.
NUREG-1150 [Keeney & al, 1991], [Cojazzi & Fogli, 2000]
This method was developed in 1987-1990 by US-NRC (Nuclear Regulatory Commission), USA. Highly structured, this approach includes training of the experts, review of discussions, individual elicitations, composition and aggregation of the opinions and review by experts. In the NUREG-1150 approach, the domain experts write reports on the issue and their final estimates are elicited individually after expert’s discussions, then averaged on an equal weight basis.
STUK-VTT [Pulkkinen & Holmberg, 1997], [Simola & Virolainen, 2000], [Cojazzi & Fogli, 2000]
This method was developed in 1997 by VTT Automation, STUK, Finland. The method is a simplification of NUREG-1150. It was originally intended for use in various kinds of quantitative risk and reliability assessments, and in engineering and economical analyses, where remarkable uncertainties are present. The use of belief networks allows an adaptation of the elicitation efforts according to the available resources. The method is based on probabilistic representation of uncertainties. The predictions obtained from experts are expressed as probability distributions. The combination of these assessments is based on hierarchical Bayes models (belief networks). Due to this property, it is also possible to deal with experts who are not familiar with the concepts of probability. Although, there are no restrictions as to the applicability of the method, it is at its best when applied to generate predictions to quantitative parameters.
SEJ-TUD [Cooke & Goossens,1996] , [Cooke & Goossens, 2000]
This procedure guide for structured expert judgement was developed in 2000 by Delft University of Technology, The Netherlands. This is a European Guide for Expert Judgement in Uncertainty Analysis. It deals with procedures to perform an expert judgement study with the aim of achieving uncertainty distributions for an uncertainty analysis. In that field of application, the methods developed at the Delft University of Technology have benefited from experience gained with expert judgement in the US with the NUREG-1150 method. The procedure guide represents a mix of these developments.
LCM [Sliter & al, 2000], [Sliter & al, 2001], [Sliter & Gregor, 2001], [Sliter, 2003]
This method was developed by EPRI, USA as part of the Life Cycle Management/Nuclear Asset Management studies. In order to guarantee long-term equipment reliability risk in nuclear power plants, LCM helps managing ageing degradation and obsolescence of important systems, structures and components. It gives an optimal solution for life cycle management based on an economical comparison between the different possible solutions.
PMDA-PIRT: Proactive Materials Degradation Assessment - Phenomena Identification and Ranking Table [NRC, 2004a], [NRC, 2004b]
This is a method in the course of development since 2003 by the US-NRC (Nuclear Regulatory Commission). It is a method for anticipating degradation mechanisms related to equipment aging in nuclear plants.
The objectives of the method are to:
- identify and rank the degradation mechanisms likely to affect critical plant equipment,
- evaluate the efficiency of periodic inspections or implement new procedures for corrective maintenance or replacement,
- and enable improved understanding of observed, but also potential degradation mechanisms.
RIPBR, Risk-Informed, Performance-Based Regulation [Golay, 2000]
RIPBR was developed at the Department of Nuclear Engineering, MIT. It is an evolving alternative to the current prescriptive method of nuclear safety regulation. RIPBR is goals oriented while the prescriptive method is means oriented. RIPBR is capable of justifying simultaneous safety and economic nuclear power improvements. It includes the formulation of probabilities through expert elicitation and the review of risk-informed, performance-based engineering analyses used to evaluate proposed changes to existing technical specifications.
TRIZ-AFD [Kaplan & al, 1999], [Guarnieri & Haïk, 2002]
This method was developed in 1997 by KAPLAN. S. & al, USA. Il allows identification and analyses of failures based on the TRIZ method. AFD ( Anticipatory Failure Determination) was recently developed in the United States. AFD consists of two tools: AFD 1 and AFD 2. AFD 1 is used to analyse failure causes. AFD 2 completes AFD1 with a number of steps for failure anticipation.
http://www.google.com/url?sa=t&source=web&cd=13&ved=0CBsQFjACOAo&url=http%3A%2F%2Fwww.knowledgeboard.com%2Fdownload%2F2471%2FState-of-the-art-on-expert-judgement-methods.doc&rct=j&q=best%20tool%20in%20expert%20judgment%3F&ei=flERTfOPAcTzcfHM9b4K&usg=AFQjCNE3t98NKTXSGzRI5KdrgxPjMZvpZQ&cad=rja
http://unfccc.int/files/adaptation/methodologies_for/vulnerability_and_adaptation/application/pdf/expert_judgment.pdf
No comments:
Post a Comment