Development of a Quality-Assessment Tool for Experimental Bruxism Studies: Reliability and Validity

Aims: To combine empirical evidence and expert opinion in a formal consensus method in order to develop a quality-assessment tool for experimental bruxism studies in systematic reviews. Methods: Tool development comprised five steps: (1) preliminary decisions, (2) item generation, (3) face-validity assessment, (4) reliability and discriminitive validity assessment, and (5) instrument refinement. The kappa value and phi-coefficient were calculated to assess inter-observer reliability and discriminative ability, respectively. Results: Following preliminary decisions and a literature review, a list of 52 items to be considered for inclusion in the tool was compiled. Eleven experts were invited to join a Delphi panel and 10 accepted. Four Delphi rounds reduced the preliminary tool-Quality-Assessment Tool for Experimental Bruxism Studies (Qu-ATEBS)- to 8 items: study aim, study sample, control condition or group, study design, experimental bruxism task, statistics, interpretation of results, and conflict of interest statement. Consensus among the Delphi panelists yielded good face validity. Inter-observer reliability was acceptable (k = 0.77). Discriminative validity was excellent (phi coefficient 1.0; P < .01). During refinement, 1 item (no. 8) was removed. Conclusion: Qu-ATEBS, the seven-item evidence-based quality assessment tool developed here for use in systematic reviews of experimental bruxism studies, exhibits face validity, excellent discriminative validity, and acceptable inter-observer reliability. Development of quality assessment tools for many other topics in the orofacial pain literature is needed and may follow the described procedure.

bruxism; Delphi technique; masticatory muscles; pain measurement

1. Glasziou P, Irwig L, Bain C, Colditz G. Systematic reviews in health care: A practical guide. Cambridge: Cambridge University Press, 2001.

2. List T, Axelsson S, Leijon G. Pharmacologic interventions in the treatment of temporomandibular disorders, atypical facial pain, and burning mouth syndrome. A qualitative systematic review. J Orofac Pain 2003;17:301–310.

3. Racine M, Tousignant-Laflamme Y, Kloda LA, Dion D, Dupuis G, Choiniere M. A systematic literature review of 10 years of research on sex/gender and experimental pain perception—Part 1: Are there really differences between women and men? Pain 2012;153:602–618.

4. Manfredini D, Guarda-Nardini L, Winocur E, Piccotti F, Ahlberg J, Lobbezoo F. Research diagnostic criteria for temporomandibular disorders: A systematic review of axis I epidemiologic findings. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2011;112:453–462.

5. Jung A, Shin BC, Lee MS, Sim H, Ernst E. Acupuncture for treating temporomandibular joint disorders: A systematic review and meta-analysis of randomized, sham-controlled trials. J Dent 2011;39:341–350.

6. Manfredini D, Lobbezoo F. Relationship between bruxism and temporomandibular disorders: A systematic review of literature from 1998 to 2008. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2010;109:e26–50.

7. List T, Axelsson S. Management of TMD: evidence from systematic reviews and meta-analyses. J Oral Rehabil 2010;37:430–451.

8. Manfredini D, Lobbezoo F. Role of psychosocial factors in the etiology of bruxism. J Orofac Pain 2009;23:153–166.

9. Charlton E. Ethical guidelines for pain research in humans. Committee on Ethical Issues of the International Association for the Study of Pain. Pain 1995;63:277–278.

10. Staahl C, Drewes AM. Experimental human pain models: A review of standardised methods for preclinical testing of analgesics. Basic Clin Pharmacol Toxicol 2004;95:97–111.

11. Svensson P. What can human experimental pain models teach us about clinical TMD? Arch Oral Biol 2007; 52:391–394.

12. Manfredini D, Marini M, Pavan C, Pavan L, Guarda-Nardini L. Psychosocial profiles of painful TMD patients. J Oral Rehabil 2009;36:193–198.

13. Madland G, Feinmann C, Newman S. Factors associated with anxiety and depression in facial arthromyalgia. Pain 2000;84:225–232. A

14. Manfredini D, Winocur E, Ahlberg J, Guarda-Nardini L, Lobbezoo F. Psychosocial impairment in temporomandibular disorders patients. RDC/TMD axis II findings from a multicentre study. J Dent 2010;38:765–772.

15. Torisu T, Wang K, Svensson P, De Laat A, Fujii H, Arendt-Nielsen L. Effects of muscle fatigue induced by low-level clenching on experimental muscle pain and resting jaw muscle activity: Gender differences. Exp Brain Res 2006;174:566–574.

16. Torisu T, Wang K, Svensson P, De Laat A, Fujii H, Arendt-Nielsen L. Effect of low-level clenching and subsequent muscle pain on exteroceptive suppression and resting muscle activity in human jaw muscles. Clin Neurophysiol 2007;118:999–1009.

17. Glaros AG, Burton E. Parafunctional clenching, pain, and effort in temporomandibular disorders. J Behav Med 2004;27:91–100.

18. Svensson P, Burgaard A, Schlosser S. Fatigue and pain in human jaw muscles during a sustained, low-intensity clenching task. Arch Oral Biol 2001;46:773–777.

19. Arima T, Svensson P, Arendt-Nielsen L. Experimental grinding in healthy subjects: A model for postexercise jaw muscle soreness? J Orofac Pain 1999;13:104–114.

20. Glaros AG, Tabacchi KN, Glass EG. Effect of parafunctional clenching on TMD pain. J Orofac Pain 1998;12:145–152.

21. Glaros AG, Baharloo L, Glass EG. Effect of parafunctional clenching and estrogen on temporomandibular disorder pain. Cranio 1998;16:78–83.

22. Plesh O, Curtis DA, Hall LJ, Miller A. Gender difference in jaw pain induced by clenching. J Oral Rehabil 1998;25:258–263.

23. Scott DS, Lundeen TF. Myofascial pain involving the masticatory muscles: An experimental model. Pain 1980; 8:207–215.

24. Bowley JF, Gale EN. Experimental masticatory muscle pain. J Dent Res 1987;66:1765–1769.

25. Christensen LV. Jaw muscle fatigue and pains induced by experimental tooth clenching: A review. J Oral Rehabil 1981; 8:27–36.

26. Clark GT, Adler RC, Lee JJ. Jaw pain and tenderness levels during and after repeated sustained maximum voluntary protrusion. Pain 1991;45:17–22.

27. Streiner DL, Norman GR. Health Measurement Scales—A practical guide to their development and use, ed 4. Oxford, England: Oxford University Press, 2008.

28. Svensson P, Arendt-Nielsen L. Effects of 5 days of repeated submaximal clenching on masticatory muscle pain and tenderness: An experimental study. J Orofac Pain 1996;10:330–338.

29. Arima T, Svensson P, Arendt-Nielsen L. Capsaicin-induced muscle hyperalgesia in the exercised and non-exercised human masseter muscle. J Orofac Pain 2000;14:213–223.

30. Glaros AG, Forbes M, Shanker J, Glass EG. Effect of parafunctional clenching on temporomandibular disorder pain and proprioceptive awareness. Cranio 2000;18:198–204.

31. Hedenberg-Magnusson B, Brodda Jansen G, Ernberg M, Kopp S. Effects of isometric contraction on intramuscular level of neuropeptide Y and local pain perception. Acta Odontol Scand 2006;64:360–367.

32. Schmidt RC. Managing Delphi surveys using non-parametric statistical techniques. Decision Sciences 1997;28:763–774; 1997.

33. Christensen LV. Some subjective-experiential parameters in experimental tooth clenching in man. J Oral Rehabil 1979;6:119–136.

34. Christensen LV. Influence of muscle pain tolerance on muscle pain threshold in experimental tooth clenching in man. J Oral Rehabil 1979;6:211–217.

35. Christensen LV. Progressive jaw muscle fatigue of experimental tooth clenching in man. J Oral Rehabil 1981; 8:413–420.

36. Christensen LV, Mohamed SE, Harrison JD. Delayed onset of masseter muscle pain in experimental tooth clenching. J Prosthet Dent 1982;48:579–584.

37. Delcanho RE, Kim YJ, Clark GT. Haemodynamic changes induced by submaximal isometric contraction in painful and non-painful human masseter using near-infrared spectroscopy. Arch Oral Biol 1996;41:585–596.

38. Farella M, Soneda K, Vilmann A, Thomsen CE, Bakke M. Jaw muscle soreness after tooth-clenching depends on force level. J Dent Res 2010;89:717–721.

39. Cohen DS, Colliver JA, Marcy MS, Fried ED, Swartz MH. Psychometric properties of a standardized-patient checklist and rating-scale form used to assess interpersonal and communication skills. Acad Med 1996;71:S87–S89.

40. Norcini JJ, Diserens D, Day SC, et al. The scoring and reproducibility of an essay test of clinical judgment. Acad Med 1990;65:S41–S42.

41. Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med 1998;73:993–997.

42. Rubin HR, Redelmeier DA, Wu AW, Steinberg EP. How reliable is peer review of scientific abstracts? Looking back at the 1991 Annual Meeting of the Society of General Internal Medicine. J Gen Intern Med 1993;8:255–258.

43. SBU. Methods of treating chronic pain. In: Axelsson S, Boivie J, Eckerlund I, et al (eds). SBU-report no 177:1. Stockholm: The Swedish Council on Technology Assessment in Health and Care (SBU), 2006.

44. Clarke M, Oxman AD (eds). Cochrane Reviewers Handbook 4.0. In Review Manager (computer programme), version 4.0. Oxford, England: The Cochrane Collaboration, 1999.

45. Moncrieff J, Churchill R, Drummond DC, McGuire H. Development of a quality assessment instrument for trials of treatments for depression and neurosis. Int J Methods Psychiatr Res 2001;10:126–133.

46. Mulrow CD. The medical review article: State of the science. Ann Intern Med 1987;106:485–488.

47. Chen CY, Palla S, Erni S, Sieber M, Gallo LM. Nonfunctional tooth contact in healthy controls and patients with myogenous facial pain. J Orofac Pain 2007;21:185–193.

48. Huang GJ, LeResche L, Critchlow CW, Martin MD, Drangsholt MT. Risk factors for diagnostic subgroups of painful temporomandibular disorders (TMD). J Dent Res 2002;81:284–288.

49. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 1996;17:1–12.

50. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: A tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003;3:25.

51. John MT. Improving TMD classification using the Delphi technique. J Oral Rehabil 2010;37:766–770.

52. Lambe P, Bristow D. What are the most important nonacademic attributes of good doctors? A Delphi survey of clinicians. Med Teach 2010;32:e347–e354.

53. Zafar SY, Currow DC, Cherny N, Strasser F, Fowler R, Abernethy AP. Consensus-based standards for best supportive care in clinical trials in advanced cancer. Lancet Oncol 2012;13:e77–e82.

54. Kennedy HP. Enhancing Delphi research: Methods and results. J Adv Nurs 2004;45:504–511.

55. Kerr M. The Delphi Process. The Delphi Process 2002 City: Remote and Rural Areas Research Initiative, NHS in Scotland, 2001.

56. Robert CJ. Use of Delphi methods in higher education. Technological Forecasting and Social Change 1972;4: 173–186.

57. Keeney S, Hasson F, McKenna HP. A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud 2001;38:195–200.

58. Moore CM. Delphi technique and the mail questionnaire. In Moore CM (ed). Group Techniques for Idea Building: Applied Social Research Methods Series, vol 9. Newbury Park, CA: Sage, 1987: 50–77.

59. Goodman CM. The Delphi technique: a critique. J Adv Nurs 1987;12:729–734.

60. Jairath N, Weinstein J. The Delphi methodology (Part one): A useful administrative approach. Can J Nurs Adm 1994;7:29–42.

61. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32: 1008–1015.

62. Reid NG. The Delphi technique: Its contribution to the evaluation of professional practice. In Ellis R (ed). Professional Competence and Quality Assurance in the Caring Professions. Beckenham, Kent, UK: Croom-Helm, 1988.

63. Baker J, Lovell K, Harris N. How expert are the experts? An exploration of the concept of ‘expert’ within Delphi panel techniques. Nurse Res 2006;14:59–70.

64. Berquez AE, Cook FM, Millard SK, Jarvis E. The Stammering Information Programme: A Delphi study. J Fluency Disord 2011;36:206–221.

65. Crisp J, Pelletier D, Duffield C, Adams A, Nagy S. The Delphi method? Nurs Res 1997;46:116–118.