From the interviews, our personal experience, and a review of documents from the Center for Medical Technology Policy, we identified five general principles that contribute to the successful engagement of stakeholders in comparative effectiveness research.
The principles are as follows: ensure a balanced representation of all stakeholder groups; get stakeholders to “buy in” to the process and make sure that they clearly understand their roles; provide neutral, expert facilitation of the stakeholder discussions; establish connections among the stakeholders; and keep the stakeholders engaged throughout the research process.
Balanced Representation Among All Groups
Sponsors and investigators must carefully select appropriate members of advisory or working groups to ensure that each relevant perspective is adequately represented. The IOM report on comparative effectiveness research priorities specifically lists “patients, caregivers, providers, payers, and policy makers” as the categories of people who should be engaged in this research. 4(p33)
Different Points Of View
Patients are a particularly important group to engage. Once they have agreed to participate, it is vital not to limit their input by presenting material in too technical a form. An experience at the Center for Medical Technology Policy in May 2009 underscores this lesson.
The center convened a working group of four clinicians, one health plan representative, and one patient. The group was charged with setting priorities for the development of methods to compare the effectiveness of emerging technologies in cardiology. A meeting facilitator asked the participants to rank technologies based on a defined set of criteria, including potential clinical benefit; quality of current clinical evidence; cost-effectiveness; potential for widespread acceptance based on demand for the technology from the health care community; and feasibility of completing studies.
After receiving a packet of briefing materials on each technology, the patient expressed concern about his ability to understand the complex clinical material, which would limit his contributions to the discussion. His response underscored the fact that the technical and multidisciplinary nature of clinical research makes it challenging to engage stakeholders from a wide range of educational backgrounds and personal experiences. Even clinicians and researchers who have a great deal of experience with a technology might not have a background in assessing cost-effectiveness, quality-of-life metrics, or other important elements of comparative effectiveness research.
During the actual meeting, the clinicians in academic medicine or community practice were most interested in the clinical usefulness of the technologies in question and the types of studies needed to evaluate their use in patients. The health plan representative wondered if the studies would be rigorous enough to demonstrate improved health outcomes and therefore to meet criteria for coverage by health insurance. The patient representative was most concerned about the possible improvement in quality of life offered by each intervention and the probability of success.
All of the participants agreed that costs were important, but not as much as clinical effectiveness. Disagreements mainly focused on the levels of evidence needed for adoption or coverage, as participants recognized that evidence thresholds might be different for individual decisions compared to population-based decisions. Based on the patient’s comments mentioned above, the center had prepared briefing materials that explained the clinical evidence for each technology in a standardized format, without using technical jargon. Since then, the center has prepared similar briefing materials for all patient and consumer representatives participating in comparative effectiveness projects.
Another method is to hold separate preparatory sessions with patient and consumer representatives before the full meeting of the multistakeholder group. This gives the nonspecialist participants an opportunity to ask questions and increase their level of understanding of the interventions being compared. Participants who have been prepared in this way are considerably more engaged in the full group’s discussions than are those who have not been briefed in advance.
Stakeholders’ Understanding And Acceptance Of Roles
Comparative effectiveness research is a relatively new concept that is still evolving. Stakeholders not normally involved in the design of clinical research studies—such as patients and representatives of health plans—might not understand why they are being invited to join a comparative effectiveness research project, or what they stand to gain from participation. Some researchers might also be unclear about their role. It is crucial that all the participants understand the fundamental purpose of the research and the rationale for involving stakeholders
Connections Among Stakeholders
Stakeholders in the health care arena often work in relative isolation and do not regularly communicate. The Institute of Medicine identified as a hallmark of comparative effectiveness research the vetting by researchers of their projects’ questions with representatives of health plans and payers. Yet it is rare for clinical researchers to consult regularly with those other stakeholders, and even rarer to involve patients and consumers in research design.
As a result, the Center for Medical Technology Policy has been developing a model for conducting coverage with evidence development in the private sector. As with the similar program in Medicare, an intervention would be provisionally covered for patients who enrolled in clinical trials to establish better evidence for that intervention. The goal is to bring stakeholders with different and sometimes competing interests together to find common goals and mutually beneficial arrangements. For example, the private-sector coverage-with-evidence-development project, funded by the California HealthCare Foundation, has required recruiting an advisory group that included health plan representatives, physicians, consumers, researchers, and other experts. Specifically, the comparative effectiveness research in question will evaluate the effectiveness of genetic testing to customize doses of warfarin, a blood-thinning medication.