Analysis of notation data: Reliability

The key factor in any research that uses new equipment is the repeatability and accuracy of this equipment. In most performance analysis papers the researchers are presenting systems that have been specifically designed for that experiment. It is the exception (Hughes et al., 1989; Wilson and Barnes, 1998) rather than the rule that papers presenting new systems produce evidence of systematised testing of the reliability of these new systems. A survey of papers presented in performance analysis at the first three world conferences on Science and Football (Reilly et al., 1988, 1993, 1997), the first two conferences on Science and Racket Sports (Reilly et al., 1995; Lees et al., 1998) and the first two world conferences on the Science of Notational Analysis of Sport (Hughes, 1996a) produced 67 papers that were experimental studies with notation systems. Seventy per cent of these did not mention reliability studies (see Table 9.1). A further 15 per cent used correlations to provide evidence of the consistency of the repeatability of the data produced from the systems. Bland and Altman (1986) have demonstrated that correlations alone are often an incomplete process for confirming reliability.