Research study out of University of Ljubljana, Slovenia:
Due to its nature and relatively poor definitions in the Code of points, judging of artistry may suffer from serious flaws in reliability and validity.
We have used the balance beam artistry evaluation forms given by 5 execution judges at World Championship in Tokyo 2011 to analyze reliability and validity.
Data on 194 competitors was gathered. Deductions were received by a highly variable number of competitors from separate judges in the same components of artistry.
The variability of average total artistry deduction was relatively large, ranging from 0.18 to 0.39 points.
The average correlation coefficient in total artistry deductions between all judge pairs was 0.6±0.06 and average correlation coefficient in total deductions from execution score was 0.73±0.04, p < 0.001.
Kendall’s coefficient W revealed significant systematic over- or under-rating of judges in the components of artistry of presentation, sureness of performance and variation in rhythm, but also in total artistry deductions (W values ranged from 0.05 to 0.53, p < 0.001 for all W coefficients).
We conclude that neither reliability nor validity of artistry judging was satisfactory in this analysis. Further thorough evaluations of judging performance in artistry are needed to guide accommodations and hopefully improvements in this field. …
Authors are grateful to Nelli Kim, FIG Women’s Technical Committee President who supported this study during and after the World Championship in Tokyo 2011.
Open the study as a PDF.
That was using the rules of the last quadrennial. Still, it’s my belief that different people cannot judge artistry consistently.
Instead, FIG judges should simply apply the Code as written now, deducting for errors in form, body position and rhythm. The most artistic will be rewarded. Consistently.
Gymnastics Coaching.com 