Positional Analysis as a Method to Assess the Consistency of the Expert Estimates
Various methods of assessing the consistency of the expert estimates which depend on the procedure of measuring the properties of the objects are analyzed. A possibility of using the method “Reliability and positional analysis” in assessing the degree of convergence of the expert evidences when the results of expert estimates are presented as a rating scale is demonstrated. Case study of wine tasting showed that the considered method of multivariate data analysis can be used to assess the consistency of expert assessments through the calculations of α Cronbach’s statistics as an integral numeric criterion of the reliability of the expert report. It is noted that the assessment of the expert consensus is an important issue given the fact that peer review is a subjective process which depends not only on the individual characteristics of objects to be assessed, but also on the physical and emotional state of the experts faced with a variety of random uncontrollable impacts. The analyzed method enables one to detect not only the objects characterized with the most divergent opinions of the experts, but also to fetch out the experts having the opinion inconsistent with the majority of colleages. The results of using the method in the framework of “STATISTICA” software to assess the consistency of experts estimates in wine tasting are presented.