Reliability of the Craniomandibular Index

Aims: To examine various dimensions of reliability of the Craniomandibular Index, a commonly used instrument for quantifying the severity of signs and symptoms of temporomandibular disorders. Methods: Classical psychometric theory and generalizability theory were used to assess the reliability of data obtained from a calibration study of examiners participating in a multi-site clinical trial and from a random community sample. Results: The reliability of aggregate scores formed by summing individual binary scored items was high, with intraclass correlations ranging from 0.81 to 0.88. When it was required that examiners recognize and agree upon a specific pattern of signs and symptoms exhibited by a patient, however, reliability dropped dramatically (multivariate kappas ranged from 0.26 to 0.32). A group of practicing examiners also showed limited ability to agree with the pattern of signs and symptoms identified by a “gold standard” examiner (multivariate kappas ranging from 0.25 to 0.32). Generalizability analysis failed to identify the specific sources of measurement error that played a major role in limiting reliability but demonstrated that generalizability of aggregate scores was very high. Conclusion: Methods of classical psychometric theory and generalizability theory support the conclusion that the reliability of aggregate scores is acceptably high. Individual items assessing certain aspects of jaw mobility and joint sounds are measured with poor reliability. Reliability declines when it is defined as the ability of examiners to agree among themselves upon a specific constellation of signs and symptoms or their ability to identify correctly a “correct” constellation identified by an expert examiner.

Craniomandibular Index; temporomandibular disorders; reproducibility of results; psychometrics; generalizability theory

1. Fricton JR, Schiffman EL. The craniomandibular index: Validity. J Prosthet Dent 1987;58:222–228.

2. Helkimo M. Studies on function and dysfunction of the masticatory system. Index for anamnestic and clinical dysfunction and occlusal state. Swed Dent J 1974;67:101–121.

3. Dworkin SF, LeResche L. Research diagnostic criteria for temporomandibular disorders: Review, criteria, examinations and specifications, critique. J Craniomandib Disord Facial Oral Pain 1992;6:301–355.

4. Dworkin SF, LeResche L, DeRouen T. Reliability of clinical measurement in temporomandibular disorders. Clin J Pain 1988;4:89–99.

5. Dworkin SF, LeResche L, DeRouen T, Von Korff M. Assessing clinical signs of temporomandibular disorders: Reliability of clinical examiners. J Prosthet Dent 1990;63:574–579.

6. Eriksson L, Westesson P-L, Sjoberg H. Observer performance in describing temporomandibular joint sounds. J Craniomandib Pract 1987;5:33–35.

7. Goulet J-P, Clark GT. Clinical TMJ examination methods. Calif Dent Assoc J 1990;18:25–33.

8. Goulet J-P, Clark GT, Flack VF, Liu C. The reproducibility of muscle and joint tenderness detection methods and maximum mandibular movement measurement for the temporomandibular system. J Orofacial Pain 1998;12:17–26.

9. Brennan RL. Elements of Generalizability Theory. Iowa City: American College Testing Program, 1983.

10. Crocker L, Algina J. Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston, 1986.

11. Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N. The dependability of behavioral measurements: Theory of generalizability of scores and profiles. New York: Wiley, 1972.

12. Fricton JR, Schiffman EL. Reliability of a craniomandibular index. J Dent Res 1986;65:1359–1364.

13. Dahlstrom L, Keeling SD, Fricton JR, Galloway Hilsenbeck S, Clark GM, Rugh JD. Evaluation of a training program intended to calibrate examiners of temporomandibular disorders. Acta Odontol Scand 1994;52:250–254.

14. Fleiss JL. Statistical methods for rates and proportions. New York: John Wiley and Sons, 1981.

15. Berry KJ, Mielke PW Jr. A generalization of Cohen’s kappa agreement measure to interval measurement and multiple raters. Educ Psychol Meas 1988;48:921–933.

16. Berry KJ, Mielke PW Jr. A generalized agreement measure. Educ Psychol Meas 1990;50:123–125.

17. Berry KJ, Mielke PW Jr. Measuring the joint agreement between multiple raters and a standard. Educ Psychol Meas 1997;57:527–530.

18. Bravo G, Potvin L. Estimating the reliability of continuous measures with Chronbach’s alpha or the intraclass correlation coefficient: Toward the integration of two traditions. J Clin Epidemiol 1991;44:381–390.

19. Marcoulides GA. An alternative method for estimating variance components in generalizability theory. Psychol Reports 1990;66:379–386.

20. Landis JR, Kock GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–174.

21. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull 1979;86:420–428.

22. Gilthorpe MS, Maddick IH, Petrie A. Introduction to multilevel modeling in dental research. Community Dent Health 2000;17:222–226.