Assessing Preventive and Therapeutic Interventions - Randomized Trials

Assessing Preventive and Therapeutic Interventions: Randomized Trials All who drink of this treatment recover in a short time, Except those whom it does not help, who all die, It is obvious, therefore, that it fails only in incurable cases. —Galen1 (129–c. 199 ce) LEARNING OBJECTIVES • To describe the important elements of randomized trials. • To define the purpose of randomization and of masking. Some ways of quantifying the natural history of disease and of expressing disease prognosis were discussed in Chapter 6. Our objective, both in clinical practice and in public health, is to modify the natural history of a disease so as to prevent or delay death or disability and to improve the health of the patient or the population. The challenge is to select the best available preventive or therapeutic interventions to achieve this goal. To do so, we need to carry out studies that determine whether these interventions work (efficacy or effectiveness) and are safe. The randomized trial is considered the ideal design for evaluating both the efficacy and the side effects of new forms of intervention. The notion of using a rigorous methodology to assess the efficacy of new drugs, or of any new modalities of care, is not recent. In 1883, Sir Francis Galton, the British anthropologist, wrote as follows: It is asserted by some, that men possess the faculty of obtaining results over which they have little or no direct personal control, by means of devout and earnest prayer, while others doubt the truth of this assertion. • To introduce design issues related to randomized trials, including stratified randomization, planned and unplanned crossovers, and factorial design. • To illustrate the problems posed by noncompliance in randomized trials. The question regards a matter of fact, that has to be determined by observation and not by authority; and it is one that appears to be a very suitable topic for statistical inquiry. ... Are prayers answered, or are they not? ... [D]o sick persons who pray, or are prayed for, recover on the average more rapidly than others?2 As with many pioneering ideas in science and medicine, many years were to pass before this suggestion was actually implemented. In 1965, Joyce and Welldon reported the results of a randomized double-blind trial of the efficacy of prayer.3 The findings of this study did not indicate that patients who were prayed for derived any special benefits from that prayer. However, a more recent study by Byrd4 evaluated the effectiveness of intercessory prayer in a coronary care unit population using a randomized double-blind protocol. The findings from this study suggested that prayer had a beneficial therapeutic effect. Which is correct? In this chapter and the one following, we discuss study designs that can be used for evaluating approaches to treatment and prevention and focus on the CHAPTER 10 Assessing Preventive and Therapeutic Interventions: Randomized Trials use of the randomized trial. Although the term randomized clinical trial is often used together with its acronym, RCT, the randomized trial design also has major applicability to studies outside the clinical setting, such as community-based trials. For this reason, we use the term randomized trial. To facilitate our discussion, reference is generally made to treatments and drugs; the reader should bear in mind that the principles described apply equally to assess new programs for screening and early detection, to compare different approaches to prevention, or new ways of organizing and delivering health services. Trials are essentially experiments that are under the control of the investigator. Compare this with observational studies reviewed in Chapter 7, where the investigator watches what unfolds but does not interfere. Suggestions of many of the elements that are important to randomized trials can be seen in many anecdotal descriptions of early trials. In a review of the history of clinical trials, Bull described an unintentional trial conducted by Ambroise Paré (1510–1590), a leading figure in surgery during the Renaissance.5 Paré lived at a time when the standard treatment for war wounds was the application of boiling oil. In 1537 Paré was responsible for the treatment of the wounded after the capture of the castle of Villaine. The wounded were so numerous that, he says: At length my oil lacked and I was constrained to apply in its place a digestive made of yolks of eggs, oil of roses and turpentine. That night I could not sleep at my ease, fearing that by lack of cauterization I would find the wounded upon which I had not used the said oil, dead from the poison. I raised myself early to visit them, when beyond my hope I found those to whom I had applied the digestive medicament feeling but little pain, their wounds neither swollen nor inflamed, and having slept through the night. The others to whom I had applied the boiling oil were feverish with much pain and swelling about their wounds. Then I determined never again to burn thus so cruelly the poor wounded. Although this was not a randomized trial, it was a form of unplanned trial, which has been carried out many times when a therapy thought to be the best available has been in short supply and has not been available for all of the patients who needed it. A planned trial was described by the Scottish surgeon James Lind in 1747.6 Lind became interested in scurvy, which killed thousands of British seamen each year. He was intrigued by the story of a sailor who had developed scurvy and had been put ashore on an isolated island, where he subsisted on a diet of grasses and then recovered from the scurvy. Lind conducted an experiment, which he described as follows: I took 12 patients in the scurvy on board the Salisbury at sea. The cases were as similar as I could have them ... they lay together in one place and had one diet common to them all. Two of these were ordered a quart of cider per day. ... Two others took 25 gutts of elixir vitriol. ... Two others took two spoonfuls of vinegar. ... Two were put under a course of sea water. ... Two others had two oranges and one lemon given them each day. ... Two others took the bigness of nutmeg. The most sudden and visible good effects were perceived from the use of oranges and lemons, one of those who had taken them being at the end of 6 days fit for duty. ... The other ... was appointed nurse to the rest of the sick. Interestingly, the idea of a dietary cause of scurvy proved unacceptable in Lind’s day. Only 47 years later did the British Admiralty allow the experiment to be repeated—this time on an entire fleet of ships. The results were so dramatic that, in 1795, the Admiralty made lemon juice a required part of the standard diet of British seamen and later changed this to lime juice. Scurvy essentially disappeared from British sailors, who, even today, are referred to as “limeys.” Randomized trials can be used for many purposes. They can be used for evaluating new drugs and other treatments of disease, including lifestyle interventions and tests of new health and medical care technology. The basic design of a randomized trial is shown in Fig. 10.1. We often begin with a defined population in which participants are randomized to receive either a new treatment or the current treatment (often referred to as “usual care” or “standard of care”), and we then follow the subjects in each group to see how many are improved in the new treatment group compared with how many are improved in the current treatment group. If the new treatment is associated with a better outcome, we would expect to find better outcomes in more of the SECTION II Using Epidemiology to Identify the Cause of Disease STUDY POPULATION RANDOMLY ASSIGNED NEW TREATMENT IMPROVE DO NOT IMPROVE CURRENT TREATMENT IMPROVE DO NOT IMPROVE Fig. 10.1 Design of a randomized trial. new treatment group than in the current treatment group. We may choose to compare two groups receiving different therapies, or we may compare more than two groups. Although at times a new treatment may be compared with no treatment, often a decision is made not to use an untreated group. For example, if we wanted to evaluate a newly developed therapy for acquired immunodeficiency syndrome (AIDS), would we be willing to have a group of AIDS patients in our study who were untreated? The answer is clearly no; we would compare the newly developed therapy with a currently recommended regimen, which would clearly be much better than no therapy at all. Perhaps the use of an untreated group may only be justified if there are not approved therapies for the disease being studied. In this case, a “placebo” can be used. Let us now turn to some of the issues that must be considered in the design of randomized trials. Chief among them is specification of the study “arms,” or treatments. These must be clearly stated with criteria for their measurement, as well as the duration of the treatments and how long the study will last. First, let’s start with who is eligible to be studied. SELECTION OF SUBJECTS The criteria for determining who will or will not be included in the study must be spelled out with great precision and in writing before the study is begun. Similar to a study hypothesis, these should be prespecified (also called a priori) at the time of the study design planning. An excellent test of the adequacy of these written criteria is to ask: If we have spelled out our criteria in writing and someone not involved in the study walks in off the street and applies our criteria to the same population, will that person select the same subjects whom we would have selected? There should be no element of subjective decision-making on the part of the investigator in deciding who is included or not included in the study. Any study procedure must in principle be replicable by others, just as is the case with laboratory experiments. Clearly, this is easier said than done, because in randomized trials we are often dealing with relatively large populations. The principle is nevertheless important, and the selection criteria must therefore be precisely stated. ALLOCATING SUBJECTS TO TREATMENT GROUPS WITHOUT RANDOMIZATION Before discussing the process of randomization, let us ask whether there might be some alternatives to randomization that could be used. Studies Without Comparison The first possible alternative is the case study or case series (as was presented in Chapter 7). In this type of study, no comparison is made with an untreated group or with a group that is receiving some other treatment. The following story was told by Dr. Earl Peacock when he was chairman of the Department of Surgery at the University of Arizona: One day when I was a junior medical student, a very important Boston surgeon visited the school and delivered a great treatise on a large number of patients who had undergone successful operations for vascular reconstruction. At the end of the lecture, a young student at the back of the room timidly asked, “Do you have any controls?” Well, the great surgeon drew himself up to his full height, hit the desk, and said, “Do you mean did I not operate on half of the patients?” The hall grew very quiet then. The voice at the back of the room very hesitantly replied, “Yes, that’s what I had in mind.” Then the visitor’s fist really came down as he thundered, “Of course not. That would have doomed half of them to their death.” God, it was quiet then, and one could scarcely hear the small voice ask, “Which half?”7 The issue of comparison is important because we want to be able to derive a causal inference regarding the CHAPTER 10 Assessing Preventive and Therapeutic Interventions: Randomized Trials relationship of a treatment and subsequent outcome. The problem of inferring a causal relationship from a sequence of events without any comparison is demonstrated in a story cited by Ederer.8 During World War II, rescue workers, digging in the ruins of an apartment house blown up in the London blitz, found an old man lying naked in a bathtub, fully conscious. He said to his rescuers, “You know, that was the most amazing experience I ever had. When I pulled the plug and the water started down the drain, the whole house blew up.” The problem exemplified by this story is: If we administer a drug and the patient improves, can we attribute the improvement to the administration of that drug? Professor Hugo Muensch of Harvard University articulated his Second Law: “Results can always be improved by omitting controls.”9 For example, in June 2020, in the first peak of the COVID-19 pandemic, results of a study were published,10 titled “Safety Update: COVID-19 Convalescent Plasma in 20,000 Hospitalized Patients.” The paper was instrumental in the Food and Drug Administration (FDA)´s decision to issue the Emergency Use Authorization for convalescent plasma for hospitalized COVID-19 patients. The authors stated that the objective of their study was to provide an update on key safety metrics after transfusion of convalescent plasma in hospitalized COVID-19 patients. They analyzed data from a convenience sample of the US FDA Expanded Access Program for COVID-19 convalescent plasma in which ALL of the 20,000 hospitalized patients with COVID-19 were transfused with convalescent plasma. That was a critical flaw of the study—the absence of a controlled comparison group who were not exposed to convalescent plasma. Despite this, the authors concluded that “These updated data provide robust evidence that transfusion of convalescent plasma is safe in hospitalized patients with COVID-19, and support the notion that earlier administration of plasma within the clinical course of COVID-19 is more likely to reduce mortality.” Unfortunately, this was a conclusion that did not align with the kind of evidence that came from such a single arm non-controlled study and was the basis of a letter to the editor criticizing the methodology and the resulting flawed conclusion.11 Since this paper included all exposed patients, it is neither a cohort study not a clinical trial; it is simply a large case series of 20,000 patients from which we cannot draw a causal inference. Studies With Comparison If we therefore recognize the need for our study to include some type of comparison, what are the possible designs? Historical Controls We could use a comparison group from the past, called historical controls. Some refer to them as external controls or synthetic controls. We have a therapy today that we believe will be quite effective, and we would like to test it in a group of patients; we know that we need a comparison group. So, for comparison, we will go back to the records of patients with the same disease who were treated before the new therapy became available. This type of design seems inherently simple and attractive. What are the problems in using historical controls? First, if today we decide to carry out the study just described, we may set up a very meticulous system for data collection from the patients currently being treated. But, of course, we cannot do that for the patients who were treated in the past, for whom we must abstract data from medical records which are likely useful for managing individual care but are fraught with error and omissions when used for research purposes. Consequently, if at the end of the study we find a difference in outcome between patients treated in the early period (historical controls) and patients treated in the later (current) period, we will not know whether there was a true difference in outcome or whether the observed difference was due only to a difference in the quality of the data collection. The data obtained from the study groups must be comparable in kind and quality; in studies using historical controls, this is often not the case. The second problem is that if we observe a difference in outcome between the early group and the later group, we will not be sure that the difference is due to the therapy because many things other than the therapy change over calendar time (e.g., ancillary supportive therapy, living conditions, nutrition, and lifestyles). This is often referred to as “secular changes.” Hence, if we observe a difference and if we have ruled out differences in data quality as the reason for the observed difference, we will not know whether the difference is a result of the drug we are studying or of other changes that take place in many other factors that may be associated with the outcome over calendar time. This is why when an investigator chooses historical controls, careful consideration should be made to whether there were major secular changes in accuracy of diagnosis, other treatments or management of the patient population being studied.