Differential Item Functioning of the Jaw Functional Limitation Scale

Aims: To assess the differential item functioning (DIF) of the Jaw Functional Limitation Scale (JFLS) due to gender, age, and language (English vs Spanish). Methods: JFLS data were collected from a consecutive sample of 2,115 adult dental patients from HealthPartners dental clinics in Minnesota. Participants with missing data were excluded, and analyses were performed using data from 1,678 participants. Whether the item response theory (IRT) model assumptions of essential unidimensionality and local independence held up for the JFLS was examined. Then, using Samejima’s graded response model, the IRT log-likelihood ratio approach was used to detect DIF. The magnitude and impact of DIF based on Raju’s noncompensatory DIF (NCDIF) cutoff value of 0.096, Cohen’s effect sizes, and test (or scale) characteristic curves were also assessed. Results: Essential unidimensionality was confirmed, but locally dependent items were found on the JFLS. A few items were flagged with statistically significant DIF after adjustment for multiple comparisons. The NCDIF indices associated with all DIF items were < 0.096, and they had small effect sizes of ≤ 0.2. The differences between the expected scores shown in the test characteristic curves were little to none. Conclusion: The present results support the use of the JFLS summary score to obtain psychometrically robust score comparisons across English- and Spanish-speaking, male and female, and younger and older dental patients. Overall, the magnitude of DIF was relatively small, and the practical impact minimal.

differential item functioning; item response theory; jaw functional limitation scale; oral health; patient-reported outcome measures

1. Oghli I, List T, John MT, Häggman-Henrikson B, Larsson P. Prevalence and normative values for jaw functional limitations in the general population in Sweden. Oral Dis 2019;25:580–587.

2. Mittal H, John MT, Sekulic´ S, Theis-Mahon N, Rener-Sitar K. Patient-reported outcome measures for adult dental patients: A systematic review. J Evid Based Dent Pract 2019;19:53–70.

3. Ohrbach R, Granger C, List T, Dworkin S. Preliminary development and validation of the jaw functional limitation scale. Community Dent Oral Epidemiol 2008;36:228–236.

4. Ohrbach R, Larsson P, List T. The jaw functional limitation scale: Development, reliability, and validity of 8-item and 20-item versions. J Orofac Pain 2008;22:219–230.

5. Schiffman E, Ohrbach R, Truelove E, et al. Diagnostic criteria for temporomandibular disorders (DC/TMD) for clinical and research applications: Recommendations of the International RDC/TMD Consortium Network and Orofacial Pain Special Interest Group. J Oral Facial Pain Headache 2014;28:6–27.

6. Xu L, He Y, Fan S, Cai B, Fang Z, Dai K. Validation of a Chinese version of the Jaw Functional Limitation Scale in relation to the diagnostic subgroup of temporomandibular disorders. J Oral Rehabil 2020;47:1–8.

7. Lövgren A, Österlund C, Ilgunas A, Lampa E, Hellström F. A high prevalence of TMD is related to somatic awareness and pain intensity among healthy dental students. Acta Odontol Scand 2018;76:387–393.

8. Kapos FP, Look JO, Zhang L, Hodges JS, Schiffman EL. Predictors of long-term temporomandibular disorder pain intensity: An 8-year cohort study. J Oral Facial Pain Headache 2018;32:113–122.

9. Gregorich SE. Do self-report instruments allow meaning-ful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Med Care 2006;44(11 suppl 3):s78–s94.

10. PROMIS Cooperative Group. PROMIS® instrument develop-ment and validation scientific standards version 2.0. 2013:1-72. http://www.nihpromis.org/Documents/PROMISStandards_ Vers2.0_Final.pdf?AspxAutoDetectCookieSupport=1.

11. Spanish Speaking Countries 2022. World Population Review. Accessed September 1, 2022. http://worldpopulationreview. com/countries/spanish-speaking-countries/

12. Harachi TW, Choi Y, Abbott RD, Catalano RF, Bliesner SL. Examining equivalence of concepts and measures in diverse samples. Prev Sci 2006;7:359–368.

13. Simancas-Pallares M, John MT, Prodduturu S, Rush WA, Enstad CJ, Lenton P. Development, validity, and reliability of the Orofacial Esthetic Scale - Spanish version. J Prosthodont Res 2018;62:456–461.

14. Simancas-Pallares M, John MT, Enstad C, Lenton P. The Spanish language 5-item oral health impact profile. Int Dent J 2020;70:127–135.

15. Dworkin SF, LeResche L. Research diagnostic criteria for temporomandibular disorders: Review, criteria, examinations and specifications, critique. J Craniomandib Disord 1992;6:301–355.

16. Stegenga B, de Bont LG, de Leeuw R, Boering G. Assessment of mandibular function impairment associated with temporo-mandibular joint osteoarthrosis and internal derangement. J Orofac Pain 1993;7:183–195.

17. Andrich D. Distinctive and incompatible properties of two common classes of IRT models for graded responses. Appl Psychol Meas 1995;19:101–119.

18. Pattanaik S, John M, Chung S, Keller S. Comparison of two rating scales with the Orofacial Esthetic Scale and practical recommendations for its application. Health Qual Life Outcomes;2022:20:131.

19. Jacques E. (2020, January 5). 11 common types of pain scales. Verywell Health. Updated August 23, 2022. Accessed September 1, 2022. https://www.verywellhealth.com/pain-scales-assessment-tools-4020329

20. Lord FM, Novick MR, Birnbaum A. Statistical Theories of Mental Test Scores. Reading, MA: Addison-Wesley, 1968.

21. Yang FM, Kao ST. Item response theory for measurement validity. Shanghai Arch Psychiatry 2014;26:171–177.

22. Browne MW. Fitting the factor analysis model. Psychometrika 1969;34:375–394.

23. Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate Data Analysis, ed 8. Andover, UK: Cengage Learning, 2018.

24. Crins MHP, van der Wees PJ, Klausch T, van Dulmen SA, Roorda LD, Terwee CB. Psychometric properties of the PROMIS Physical Function item bank in patients receiving physical therapy. PLoS One 2018;13:e0192187.

25. Maydeu-Olivares A, Joe H. Assessing approximate fit in categorical data analysis. Multivariate Behav Res 2014;49:305–328.

26. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS (eds). Testing Structural Equation Models. Newbury Park, CA: Sage, 1993:136–162.

27. Tucker LR, Lewis C. A reliability coefficient for maximum likeli-hood factor analysis. Psychometrika 1973;38:1–10.

28. Cai L, Chung SW, Lee T. Incremental model fit assessment in the case of categorical data: Tucker-Lewis Index for item response theory modeling. Prev Sci 2021. Epub ahead of print May 10.

29. Joe H, Maydeu-Olivares A. A general family of limited information goodness-of-fit statistics for multinomial data. Psychometrika 2010;75:393–419.

30. Chalmers RP. Mirt: A multidimensional item response theory package for the R environment. J Stat Software 2012;48:1–29.

31. Chen WH, Thissen D. Local dependence indexes for item pairs using item response theory. J Educ Behav Stat 1997;22:265–289.

32. Edelen MO, Reeve BB. Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Qual Life Res 2007;16(suppl 1):5–18.

33. Samejima F. Graded response model. In: van der Linden WJ, Hambleton RK (eds). Handbook of Modern Item Response Theory. New York, NY: Springer New York, 1997:85–100.

34. Teresi JA, Ocepek-Welikson K, Kleinman M, et al. Analysis of differential item functioning in the depression item bank from the Patient Reported Outcome Measurement Information System (PROMIS): An item response theory approach. Psychol Sci Q 2009;51:148–180.

35. Nguyen TH, Han HR, Kim MT, Chan KS. An introduction to item response theory for patient-reported outcome measurement. Patient 2014;7:23–35.

36. Flowers CP, Oshima TC, Raju NS. A description and demonstration of the polytomous-DFIT framework. Appl Psychol Meas 1999;23:309–326.

37. Orlando Edelen MO, Thissen D, Teresi JA, Kleinman M, Ocepek-Welikson K. Identification of differential item functioning using item response theory and the likelihood-based model comparison approach. Application to the Mini-Mental State Examination. Med Care 2006;44(11 suppl 3):s134–s142.

38. Langer MM, Hill CD, Thissen D, Burwinkle TM, Varni JW, DeWalt DA. Item response theory detected differential item functioning between healthy and ill children in quality-of-life measures. J Clin Epidemiol 2008;61:268–276.

39. Teresi JA. Different approaches to differential item functioning in health applications. Advantages, disadvantages and some neglected topics. Med Care 2006;44(11 suppl 3):s152–s170.

40. Benjamini Y, Hochberg Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. J Royal Stat Soc: Series B (Methodological) 1995;57:289–300.

41. Teresi JA, Ramirez M, Lai JS, Silver S. Occurrences and sources of Differential Item Functioning (DIF) in patient-reported outcome measures: Description of DIF methods, and review of measures of depression, quality of life and general health. Psychol Sci Q 2008;50:538.

42. Teresi JA, Ocepek-Welikson K, Ramirez M, Fieo R, Fulmer T, Gurland BJ. Development of a short-form of the medication management test: Evaluation of dimensionality, reliability, information and measurement equivalence using latent variable models. J Nurs Meas 2018;26:483–501.

43. Raju NS, van der Linden WJ, Fleer PF. IRT-based internal measures of differential functioning of items and tests. Appl Psychol Meas 1995;19:353–368.

44. Collins WC, Raju NS, Edwards JE. Assessing differential functioning in a satisfaction scale. J Appl Psychol 2000;85:451–461.

45. Cohen J. Statistical Power Analysis for the Behavioral Sciences, ed 2. Hillsdale, NJ: Erlbaum, 1988.

46. Hays RD, Calderón JL, Spritzer KL, Reise SP, Paz SH. Differential item functioning by language on the PROMIS physical functioning items for children and adolescents. Qual Life Res 2018;27:235–247.

47. StataCorp. Statistical Software: Release 14. College Station, TX: StataCorp LP, 2015.

48. Cervantes VH. DFIT: An R package for Raju’s differential functioning of items and tests framework. J Stat Software 2017;76:1–24.

49. John MT. Health outcomes reported by dental patients. J Evid Based Dent Pract 2018;18:332–335.

50. Olde Rikkert MGM, van der Wees PJ, Schoon Y, Westert GP. Using patient reported outcomes measures to promote inte-grated care. Int J Integr Care 2018;18:8.

51. Walker CM. What’s the DIF? Why differential item functioning analyses are an important part of instrument development and validation. J Psychol Assess 2011;29:364–376.

52. Pattanaik S, John MT, Chung S. Assessment of differential item functioning across English and Spanish versions of the Orofacial Esthetic Scale. J Oral Rehabil 2021;48:73–80.

53. Campos LA, Marôco J, John MT, Santos-Pinto A, Campos JADB. Development and psychometric properties of the Portuguese version of the Orofacial Esthetic Scale: OES-Pt. PeerJ 2020;8:e8814.

54. Lee J, Stucky B, Rozier G, Lee SY, Zeldin LP. Oral health literacy assessment: Development of an oral health literacy instrument for Spanish speakers. J Public Health Dent 2013;73:1–8.

55. Health Literacy in Dentistry. American Dental Association. Accessed September 1, 2022. https://www.ada.org/en/public-programs/health-literacy-in-dentistry

56. Scott NW, Fayers PM, Aaronson NK, et al. Differential item functioning (DIF) analyses of health-related quality of life instru-ments using logistic regression. Health Qual Life Outcomes 2010;8:81.

57. Kim S, Cohen A, Alagoz C, Kim S. DIF detection and effect size measures for polytomously scored items. J Educ Meas 2007;44:93–116.

58. Tay L, Meade AW, Cao M. An overview and practical guide to IRT measurement equivalence analysis. Organ Res Methods 2014;18:3–46.

59. Baker FB. The Basics of Item Response Theory, ed 2. ERIC Clearinghouse on Assessment and Evaluation, College Park, MD: 2001.

60. Pattanaik S, John MT, Kohli N, et al. Item and scale properties of the Oral Health Literacy Adults Questionnaire assessed by item response theory. J Public Health Dent 2021;81:214–223.

61. Bandalos DL. Measurement Theory and Applications for the Social Sciences. New York, NY: Guilford Press, 2018.

62. Meade AW, Lautenschlager GJ, Johnson EC. A Monte Carlo examination of the sensitivity of the differential functioning of items and tests framework for tests of measurement invariance with Likert data. Appl Psychol Meas 2007;31:430–455.

63. Bolt DM. A Monte Carlo comparison of parametric and Nonparametric Polytomous DIF detection methods. Appl Meas Educ 2002;15:113–141.

64. Meade AW. A taxonomy of effect size measures for the differential functioning of items and scales. J Appl Psychol 2010;95:728–743.

65. Nguyen HT, Clark M, Ruiz RJ. Effects of acculturation on the reporting of depressive symptoms among Hispanic pregnant women. Nurs Res 2007;56:217–223.

66. Kim BS, Lee DW, Bae JN, et al. Effects of education on differential item functioning on the 15-Item modified Korean version of the Boston Naming Test. Psychiatry Investig 2017;14:126–135.

67. Baghaei P. Local dependency and Rasch measures. Accessed September 1, 2022.. https://www.rasch.org/rmt/rmt213b.htm

68. McDonald RP. A basis for multidimensional item response theory. Appl Psychol Meas 2000;24:99–114.

69. Reissmann DR. Dental patient-reported outcome measures are essential for evidence-based prosthetic dentistry. J Evid Based Dent Pract 2019;19:1–6.

70. Hua F. Increasing the value of orthodontic research through the use of dental patient-reported outcomes. J Evid Based Dent Pract 2019;19:99–105.

71. Palaiologou A, Kotsakis GA. Dentist-patient communication of treatment outcomes in periodontal practice: A need for dental patient-reported outcomes. J Evid Based Dent Pract 2020;20:101443.

72. Listl S. Value-based oral health care: Moving forward with dental patient-reported outcomes. J Evid Based Dent Pract 2019;19:255–259.