Literature Syntheses for the Council on Chiropractic Guidelines and Practice Parameters: Methodology

Article Outline

• Abstract
• Overview of Process
• Team Selection and Orientation Training of Team Leaders
• Summary of Process
• Methods Used to Identify and Retrieve Evidence
  • Identification
  • Retrieval
• Methods to Evaluate the Evidence
  • Definitions for Evidence Ratings
    • Grade A: Supported by good evidence from relevant studies. Must be included in evidence tables and as a reference(s) for best practices
      • Explanation
      • Examples
    • Grade B: Supported by fair evidence from relevant studies. Must be included in evidence tables and as reference(s) for best practices
      • Explanation
      • Examples
    • Grade C: Supported by limited evidence from studies or reviews. Is not included in evidence tables but as reference(s) for best practices
      • Explanation
      • Examples
    • Grade I: No recommendation can be made because of insufficient or nonrelevant evidence. It should not be included in evidence tables or as reference(s) for best practices
      • Explanation
      • Examples
• Topic Selection
• Use of Evidence Tables
  • Stakeholder Review and Implementation
• Audit and Review
• Acknowledgment
• References
• Copyright

Abstract 

Objective

The purpose of this project was to initiate an iterative process for systematic review of the literature involving a broad spectrum of individuals with experience across multiple domains (clinicians, educators, clinical scientists, and politically active) within the chiropractic profession.

Methods

The Scientific Commission of the Council on Chiropractic Guidelines and Practice Parameters (CCGPP) was charged with developing literature syntheses, organized by anatomical region, to evaluate and report on the evidence-based values for chiropractic care. Content and process–experienced team leaders were selected to manage 8 domains based on regional disorders: low back and related lower extremity conditions; neck pain, headache, and related upper extremity conditions; costovertebral and thoracic conditions; upper extremity disorders; lower extremity disorders; nonmusculoskeletal disorders; and subluxation. Team efforts in review, rating, and reporting of literature synthesis were guided, as best possible, by the widely accepted Appraisal of Guidelines for Research and Evaluation process. The main features included (1) review by a panel of experts; (2) detailed topic selection based on literature of most common conditions and procedures; (3) structured instruments for rating the quality of and results from the literature; (4) formal consensus process to adjudicate differences in professional opinion; and (5) wide stakeholder review by patients, professionals, policymakers, and third-party payers. As part of the CCGPP process, preliminary drafts of these articles were posted on the CCGPP Web site www.ccgpp.org (2006-2008) to allow for an open process and the broadest possible mechanism for stakeholder input.

Results

Reports on findings from this process are being published. The reports from each domain summarize methodological challenges and their unique content.

Conclusions

Although all literature in health care is challenged by complex methodological issues that limit how the information may be generalized, the preponderance of evidence in any of the domains can be informative to the clinician as well as give guidance to new scientific efforts to improve the quality of care.

Key Indexing Terms: Practice Guideline, Chiropractic

Back to Article Outline

Overview of Process 

The basic methods of literature synthesis are well established. The material here is intended as a brief description of the manner in which the Council on Chiropractic Guidelines and Practice Parameters (CCGPP) teams were directed. Any specific variation in procedure dictated by the quality and quantity of literature encountered by each team is described in the individual sections. The CCGPP and its organizational structure are described in the accompanying article¹ describing evidence and its Web site.

Representing its constituent member organizations, the council approved a listing of disorders by International Classification of Diseases, Ninth Revision, Clinical Modification, codes that would form the scope of the conditions that were investigated. Council on Chiropractic Guidelines and Practice Parameters teams were identified, consisting of content experts from within the profession and involving consultants that are cross-trained or external to the profession in select areas. The practice of chiropractic was divided into general areas based on anatomical regions (Table 1), and the list was given to the commission who, through its teams, used it to design the literature searches within each domain. Using surveys of the profession², ³ and publications on practice audits,⁴, ⁵, ⁶ each team selected the topics for review by this first iteration. The criteria used were based on the team's determination of the most common disorders seen and most common classifications of treatments used by chiropractors based on the literature.

Table 1. Practice domains identified for grouping of similar conditions for searching the literature and reporting of best practices

Back to Article Outline

Team Selection and Orientation Training of Team Leaders 

The CCGPP council appointed 2 cochairs for the commission, each having experience in practice, structured literature review, and formal consensus processes, either having been involved with one or more of original clinical and educational research, the Agency for Health Care Policy and Research acute low back pain guidelines, the RAND corporation task forces on appropriateness for use of spinal manipulation, and earlier CCGPP Mercy Center chiropractic guidelines. Team leaders were nominated by the commission cochairs and recommended to the council for approval. Selection was based upon identification of individuals with clinical experience, additional cross-training in the content area of their assigned domain, and/or scholarly work. Team members were selected from a multidisciplinary list of practitioners and content experts that had been solicited from the council stakeholders and colleges. Additional nominees were identified to serve as consultants based on content expertise. Once a team leader accepted members for his/her team, no changes were permitted in the team composition without being initiated by the team leader to add or replace members as necessary. All changes were submitted to the council for agreement before implementation. All commission members' service was uncompensated.

A team leader packet of information, derived from the literature, sets out motivation and methodology, including standardized instruments, with example formats serving as suggestion for the final report. An orientation meeting was convened with all team leaders and available consultants at the 2004 Association of Chiropractic Colleges and Research Agenda Conference held in Las Vegas, Nev. Survey of the literature, rating, and interpretation of evidence commenced in July of 2004.

Back to Article Outline

Summary of Process 

Balancing patient-centered and evidence-based values imparts similar internal tensions with tendency for the best intent of individuals to succumb to training biases and personal preferences. Four strategies were used to minimize this problem while empowering legitimate and informed interpretation of the literature:

A schematic of the process followed in developing the conclusions for best practices is given in Figure 1. Process development was guided by experience of commission members with the RAND consensus process, Cochrane collaboration, Agency for Health Care Policy and Research, and published recommendations modified to the needs of the council.

• 
  • View Large Image
  • Download to PowerPoint

• Fig 1. 

  Council on Chiropractic Guidelines and Practice Parameters flow chart describing the process of best practices development.

Back to Article Outline

Methods Used to Identify and Retrieve Evidence 

Identification 

Topics were selected based on the most common disorders seen in chiropractic practice and most common classifications of treatments used by chiropractors based on the literature. The report for each region includes the specifics on decisions made by the team.

Retrieval 

Searches of electronic databases were conducted, supplemented by hand searches of published literature. College libraries facilitated retrieval by providing paper or electronic copies of literature to each team.

Back to Article Outline

Methods to Evaluate the Evidence 

Standardized and validated instruments used by the Scottish Intercollegiate Guidelines Network were used to evaluate the following types of studies⁷:

For guidelines, the Appraisal of Guidelines for Research and Evaluation instrument was used.⁸ For case series, the cochairs developed a checklist modified from other instruments. A standardized method for grading the strength of the evidence was used, as described below. Each team's panel conducted the review and evaluation of the evidence.

Definitions for Evidence Ratings 

The grading for strength of the evidence is described below and summarized in Figure 2.

Fig 2. Summary of grading of strength of evidence.

GRADE A: Good evidence from relevant studies.

•Studies with appropriate designs and sufficient strength to answer the questions.

•Results are both clinically important and consistent with minor exceptions at most.

•Results are free of significant doubts about generalizability, bias, and design flaws.

•Negative studies have sufficiently large sample sizes to have adequate statistical power.

GRADE B: Fair evidence from relevant studies.

•Studies of appropriate designs of sufficient strength, but inconsistencies or minor doubts about generalizability, bias and design flaws, or adequacy of sample size.

•Evidence solely from weaker designs, but confirmed in separate studies.

GRADE C: Limited evidence from studies/reviews.

•Studies with substantial uncertainty due to design flaws, or adequacy of sample size.

•Limited number of studies weak design for answering the question addressed.

GRADE I: No recommendation can be made because of insufficient or non-relevant evidence.

•No evidence that directly pertains to the addressed question either because studies have not been performed or published, or are non-relevant.

Explanation 

Examples 

Explanation 

Examples 

Explanation 

Examples 

Explanation 

Examples 

In the initial methodology, “grade D” was also included, which was defined as follows:

Because recent rating systems, such as the US Preventive Services Task Force, use a grading of “D” to indicate negative or no benefit,⁹ it was felt that use of grade D would be confusing to readers and might give a connotation of no benefit; so as a midcourse correction, use of grade D was abandoned.

Back to Article Outline

Topic Selection 

Patients having many clinical descriptions seek care from chiropractors based upon the generally recognized reputation and the individual doctor's practice focus. Some providers center specifically on subluxation and its manifestations, whereas others limit their practice to treating patients with spinal disorders or musculoskeletal complaints. Finally, others address more general health problems, prevention, and special populations. The diversity of professional practice makes the review of all related literature an impossible task. To accommodate the need for substantive review of the most relevant and informative literature, we developed an interactive process.

Back to Article Outline

Use of Evidence Tables 

Evidence tables for RCTs rated by the team were constructed using categorical information shown reliable in other studies. Templates were provided to each team member for recording this information during the course of their review.

Stakeholder Review and Implementation 

Stakeholder review of best practices is a critical step to facilitate final recommendations and implementation. This process affords the opportunity for individuals and groups that can be impacted by best practices to provide comment and documentation for consideration by the team. Stakeholders for the low back and related lower extremity symptoms are considered to include doctors of chiropractic, students and prospective students, educators and teaching institutions, professional organizations and agencies, third-party payers, governmental agencies, and patients.

Three separate strategies have been used to inform interprofessional stakeholders on progress during the development of the best practices document. By providing periodic updates, colleges, associations, and providers were made aware of the pending release for review and comment. The 3 methods included (1) periodic articles published in interprofessional news media, (2) presentations at the Association of Chiropractic Colleges, the Federation of Chiropractic Licensing Boards meetings, and (3) providing a speaker's bureau for use in presentations to state professional association meetings.

Two strategies were used to reach stakeholders for review and comment on the document itself. On completion of the draft document of best practices, a summary of the best practices document was posted on a widely accessed health care Web site (spine-health.com) that experienced a public hit rate of 2.5 to 3.0 million per month during 2005. Separately, on the CCGPP Web site, the document was posted and notification made to colleges, state and national associations, and third-party payers.

Interactive electronic questionnaires, developed by the Dissemination, Implementation, Evaluation, and Review committee of CCGPP, are available for stakeholder comments online. Those choosing to comment are invited to submit documentation for their opinions directly to CCGPP. The postings will be maintained for 60 days and comments harvested electronically and provided to the cochair of the commission. The cochair will group similar comments and develop summary questions that will be posted, with the original comments and any supportive documentation, to the team for review and response. A tally of comments by group along with the questions and responses from the team will be made a part of the Appendix in the final document release.

The final document will include any changes in conclusions of the team made in response to stakeholder input.

Back to Article Outline

Audit and Review 

As noted earlier, the best practices effort of the CCGPP is designed as an iterative process. The low back and related lower extremity best practices document is intended to be reviewed with inclusion of any new evidence and extension of the domains considered on a 2- to 5-year cycle, depending on the state of the art in the literature. (Details on Audit and Review to be completed in conference with the Dissemination, Implementation, Evaluation, and Review committee following the stakeholder review and comments.)

Back to Article Outline

Acknowledgment 

The author acknowledges the team leaders and dozens of team members, content experts, and consultants who devoted their uncompensated time and efforts on behalf of the profession.

Back to Article Outline

References 

1. Triano JJ. What constitutes evidence for best practice?. J Manipulative Physiol Ther. 2008;31:[this issue]
   • View In Article
2. McDonald WP, Durkin KF, Pfefer M. How chiropractors think and practice: the survey of North American chiropractors. Sem Integr Med. 2004;2:92–98
   • View In Article
3. Christensen M, Kollasch M, Ward R, Webb K, Day A, ZumBrunnen J. Job Analysis of Chiropractic. Greeley, CO: NBCE; 2005;
   • View In Article
4. Coulter ID, Hurwitz EL, Adams AH, Genovese BJ, Hays R, Shekelle PG. Patients using chiropractors in North America: who are they, and why are they in chiropractic care?. Spine. 2002;27:291–296[discussion 297-298]
   • View In Article
   • CrossRef
5. Coulter ID, Singh BB, Riley D, Der-Martirosian C. Interprofessional referral patterns in an integrated medical system. J Manipulative Physiol Ther. 2005;28:170–174
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (79 KB)
   • CrossRef
6. Hurwitz EL, Coulter ID, Adams AH, Genovese BJ, Shekelle PG. Use of chiropractic services from 1985 through 1991 in the United States and Canada. Am J Public Health. 1998;88:771–776
   • View In Article
   • MEDLINE
   • CrossRef
7. Scottish Intercollegiate Guidelines Network . A guideline developers' handbook. Edinburgh: SIGN; 2001;
   • View In Article
8. The AGREE Collaboration. Appraisal of guidelines for research & evaluation (AGREE) instrument. www.agreecollaboration.org
   • View In Article
9. Sawaya GF, Guirguis-Blake J, LeFevre M, Harris R, Petitti D. Update on methods. U.S. Prev Serv Task Force. 2007;
   • View In Article