Extraction of RDC/TMD Subscales from the Symptom Check List-90: Does Context Alter Respondent Behavior?

Aims: To test whether extraction of the 2 subscales in the Research Diagnostic Criteria for Temporomandibular Disorders (RDC/TMD) affected the subscale score reliability and whether scores from the RDC/TMD subscales are comparable to the same scales when the whole Symptom Check List-90 (SCL-90R) is administered. Methods: The full SCL90-R and a modified version containing only the depression and somatization scales were administered in counterbalanced order to 103 subjects. As another test of context, a subset of participants completed the modified and full versions as part of a larger battery of instruments relevant to facial pain. Statistical analyses included internal reliability for item analysis andintraclass correlation (ICC) and Lin’s concordance correlation coefficient (CCC) for total scale score reliability. Results: Internal reliability was approximately 0.95 for depression and 0.87 for somatization, independent of test form. Total scale scores were reliable across test versions, with both ICC and CCC approximately 0.95 for depression and 0.91 for somatization. Permutation tests using the CCC indicated a mild influence on the somatization score but not the depression score due to order effects, but these effects were not significant when considering the 95% CIs based on resampling methods. Conclusion: Whether items from other subscales are present or not does not affect the internal reliability or parallel forms reliability of the total scores from either depression or somatization. Context of administration, via order of forms completion, does not alter total score or reliability of depressive items but may alter total scores for somatization.

psychometrics; RDC/TMD; reliability; validity

1. Dworkin SF, LeResche L. Research Diagnostic Criteria for Temporomandibular Disorders: Review, criteria, examinations and specifications, critique. J Craniomandib Disord 1992;6:301–355.

2. McNeill C, Mohl ND, Rugh JD, et al. Temporomandibular disorders: Diagnosis, management, education, and research. J Am Dent Assoc 1990;120:253–263.

3. Von Korff M, Dworkin SF, Le Resche L, Kruger A. An epidemiologic comparison of pain complaints. Pain 1988; 32:173–183.

4. Kight M, Gatchel RJ, Wesley L. Temporomandibular disorders: Evidence for significant overlap with psychopathology. Health Psychol 1999;18:177–182.

5. Ohrbach R, LeResche L, Dworkin SF. Longitudinal changes in TMD: Influence of baseline findings and treatment-seeking. J Dent Res 1997;76:389.

6. Dworkin SF. Behavioral, emotional, and social aspects of orofacial pain. In: Stohler CS, Carlson DS (eds). Biological and Psychological Aspects of Orofacial Pain. Ann Arbor, MI: Center for Human Growth and Development, 1994: 93–112.

7. Wilson L, Dworkin SF, Whitney C, LeResche L. Somatization and pain dispersion in chronic temporomandibular disorder pain. Pain 1994;57:55–61.

8. Turner JA, Dworkin SF. Screening for psychosocial risk factors in patients with chronic orofacial pain: Recent advances. J Am Dent Assoc 2004;135:1119–1125.

9. Derogatis LR, Lipman RS, Covi L. SCL-90: An outpatient psychiatric rating scale—Preliminary report. Psychopharmacology 1973;9:13–28.

10. Von Korff M, Ormel J, Keefe FJ, et al. Grading the severity of chronic pain. Pain 1992;50:133–149.

11. Dworkin SF, Sherman J, Mancl L, Ohrbach R, LeResche L, Truelove E. Reliability, validity, and clinical utility of RDC/TMD Axis II scales: Depression, non-specific physical symptoms, and graded chronic pain. J Orofac Pain 2002;16:207–220.

12. Anastasi A. Psychological Testing. New York: Macmillan, 1988.

13. Smith GT, McCarthy DM, Anderson KG. On the sins of short-form development. Psychol Assess 2000;12: 102–111.

14. American Educational Research Association. Standards for Educational and Psychological Testing. Washington, DC: Author, 1999.

15. Embretson SE, Reise SP. Item Response Theory for Psychologists. Mahwah, NJ: Lawrence Erlbaum, 2000.

16. List T, Dworkin SF. Comparing TMD diagnoses and clinical findings at Swedish and U.S. TMD centers using Research Diagnostic Criteria for Temporomandibular Disorders. J Orofac Pain 1996;10:240–253.

17. Lobbezoo F, van Selms MKA, John MT, et al. Use of the Research Diagnostic Criteria for Temporomandibular Disorders for multinational research. Translation efforts and reliability assessments in The Netherlands. J Orofac Pain 2004;19:301–308.

18. Wahlund K, List T, Dworkin SF. Temporomandibular disorders in children and adolescents: Reliability of a questionnaire, clinical examination, and diagnosis. J Orofac Pain 1998;12:42–52.

19. Yap AU, Dworkin SF, Chua EK, List T, Tan KB, Tan HH. Prevalence of temporomandibular disorder subtypes, psychologic distress, and psychosocial dysfunction in Asian patients. J Orofac Pain 2003;17:21–28.

20. John MT, Hirsch C, Reiber T, Dworkin S. Translating the Research Diagnostic Criteria for Temporomandibular Disorders into German: Evaluation of content and process. J Orofac Pain 2006;20:43–52.

21. Campbell DT, Stanley JC. Experimental and Quasi-Experimental Designs for Research. Boston: Houghton Mifflin, 1963.

22. Derogatis LR. SCL-90-R: Administration, Scoring and Procedures Manual-II, for the Revised Version. Towson, MD: Clinical Psychometric Research, 1983.

23. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull 1979;86:420–428.

24. Bartko JJ, Carpenter WT Jr. On the methods and theory of reliability. J Nerv Ment Dis 1976;163:307–317.

25. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307–310.

26. Bland JM, Altman DG. A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two methods of measurement. Comput Biol Med 1990;20:337–340.

27. Chinn S. The assessment of methods of measurement. Stat Med 1990;9:351–362.

28. Ludbrook J. Comparing methods of measurement. Clin Exp Pharmacol Physiol 1997;24:193–203.

29. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989;45:255–268.

30. Cole JC, Rabin AS, Smith TL, Kaufman AS. Development validation of a Rasch-derived CES-D short form. Psychol Assess 2004;16:360–372.

31. Steinberg L. Context and serial-order effects in personality measurement: Limits on the generality of measuring changes the measure. J Person Soc Psychol 1994;66: 341–349.

32. Hamilton JC, Shuminsky TR. Self-awareness mediates the relationship between serial position and item reliability. J Person Soc Psychol 1990;59:1301–1307.

33. Knowles ES, Byers B. Reliability shifts in measurement reactivity: Driven by content engagement or self-engagement? J Person Soc Psychol 1996;70:1080–1090.