Bioethical, Reproducibility, and Translational Challenges of Animal Models

Abstract

There is no prescribed stage or standardized point at which an animal model protocol is reviewed for reproducibility and translatability. The method of review for a reproducible and translatable study is not consistently documented in peer literature, and this is a major challenge for those working with animal models of human diseases. If the study is ill designed, it is impossible to perform an accurate harm/benefit analysis. In addition, there may be an ethical challenge if the work is not reproducible and translatable. Animal welfare regulations and other documents of control clearly state the role of the Institutional Animal Care and Use Committees are to look at science justification within the context of animal welfare. This article, concentrating on models not governed by regulations, outlines issues and offers recommendations for refining animal model review with a goal to improve study reproducibility and translatability.

INTRODUCTION

This paper centers on bioethical, reproducibility, and translational challenges with models of human disease. Studies with animals for basic research, while very important, are not a topic for this discussion. Animals as models of human biology and disease have contributed significantly to the progress in understanding, preventing, mitigating, and managing human disease. Animals are used to provide meaningful insights into fundamental biological processes, disease, responses to injury, and even to support the development of human medical interventions such as diagnostics, vaccines, pharmaceuticals, and medical devices. It is considered unethical to conduct primary research in human patients where pain, distress, suffering, or permanent harm could be induced. That said, human experimentation happens every day in the form of clinical trials, testing novel medical interventions or empirical clinical treatments to manage an existing disease. The risks to humans are mitigated by the conduct of “preclinical” or “nonclinical” research, which is intended to provide evidence or support for the likely success and safety of a medical intervention.

Harmful human experimentations have been well documented throughout history. The most internationally notorious and famous use of humans as experimental subjects occurred during World War II in Nazi Germany. One outcome of this period was the development of the Nuremberg Code, named for the city where the trials were held.¹ The intent of the code is to ensure the moral, ethical, and legal basis for human experimentation is justified as having potential benefit to the patient and society.² In fact, principle 3 states “The (human) study should be conducted based on the results of animal experimentation and a knowledge of the natural history of the disease or other problem under study that the anticipated results will justify the performance of the experiment.” This principle of proving safety and efficacy prior to studies in humans is still fundamental in the review of human clinical trials. US Federal Regulatory standards define the form, function, and scope of institutional review boards (IRBs).^3,4 Within those standards is the requirement that “Risks to subjects are reasonable in relation to anticipated benefits, if any, to subjects and the importance of the knowledge that may be expected to result” be considered.⁵

“Anticipated benefits” for novel interventions are often supported by evidence derived from animal studies. Weaknesses in the translational relevance of those animal studies lessens the ability of IRBs to mitigate relative risk to human participants.

INSTITUTIONAL ANIMAL CARE AND USE COMMITTEES (IACUC) ASSESSMENT

An important component of the ethical review of an animal experiment is an assessment of the harm/benefit analysis (HBA); there is international guidance to develop criteria for assessment of the HBA in animal studies.^6,7 A major impediment to the HBA determination is not in the weighing of harms but in difficulty of assessing benefit beyond what is stated in the protocol submission.

We believe there is a bioethical requirement that animal studies, as defined below, be reproducible and planned with specific questions in mind. Studies should generate data to answer the questions for which they were designed. For the specifics of this article, we are looking at studies intended for direct preclinical evaluation of therapies for clinical translation. The review, design, conduct, and reporting of these “confirmatory” studies differs from those of an exploratory nature.⁸ Study validity is under scrutiny, with many reports where the results in preclinical models cannot be reproduced and/or outcomes that do not replicate outcomes in humans or both.^9–12

It is important to note reproducibility refers to the ability for others, outside a specific laboratory, center, or even person, to reproduce the findings of a published study using animal models. Translatability is the validity of the model in replicating a specific aspect of human targets, mechanisms, and/or disease.¹³

This debate has put important focus on the analytical rigor of animal study designs, the completeness of animal research reporting standards, and the biological or translational relevance of animals as models of the human condition. One consequence of this debate has been interest and investment in the discovery, development, and adoption of non-animal modeling capabilities such as microphysiological systems.^14,15 The resulting growth in the availability of human-relevant modeling alternatives should enable progression toward a decreased dependence on animals with a concomitant increase in human-based complex in vitro models.^16–18 But the complexity of mammalian biology will remain elusive to even the most sophisticated modeling systems, requiring that we still conduct scientifically defensible and judicious animal research. To ensure our moral and ethical responsibilities to both humans and animals, we need to significantly improve the human relevance of our animal research by increasing reproducibility and translatability of study design.

IS IMPROVED PRECISION POSSIBLE?

Much of the current effort and discussion related to the human or translational relevance of animal research has focused on standardization of reporting to ensure reproducibility or, in the extreme, advocating animal research be abandoned altogether. As noted above, complete abandonment of animal research is not possible if we still want to meet public expectations of improved health and treatment. Our obligation is twofold, and both are of equal priority. The first is to reduce our dependence on animals by accelerating the development and adoption of alternatives. The second is to improve the human relevance of the animal studies we must continue to do. Success in these efforts will enable an effective and timely evolution in the quest of progress human health with more precise models.

Substantive improvement in the impact of animal research requires we give more attention to the precision of the questions we ask, the selection of physiologically and mechanistically relevant animal models, the induction of human-relevant experimental disease, the design of clinically relevant studies, and the measures we collect. Aligning these features of an animal study to the human intent is critical to ensuring the translational relevance of that animal study and, in turn, must be considered in the ethical justification for conducting the study. Conducting an animal study intended to support a human or clinical intervention not optimized for human application cannot be considered an ethical use of the animals. In the same way, conducting a human clinical trial with a novel therapeutic without the benefit of “preclinical assessment” would most likely be rejected by regulators.

The roles and responsibilities of the IACUC have been well analyzed by many and will not be reviewed again here.^19,20 IACUCs discuss, debate, and decide on protocols covering a wide range of studies, from those intended for basic knowledge to studies held in the field and wild.^21–23 For the rest of this article, animal studies are discussed regarding both reproducibility and translatability. Though we are concentrating on models of translation, the method and process for IACUC review and debate is the same as other protocol submissions.²⁴ Ideally, protocols should be reviewed for both study robustness (reproducibility) and translatability, 2 different aspects of a well-designed study.²⁵ However, as noted, the legal responsibility of the IACUC is to evaluate protocols for study justification and the effect of the study on animal welfare.

As stated earlier, for models intended to be translatable to human conditions, the IACUC reviews both the predicted harm to the welfare of the animal subject and the proposed benefit to humans of the study. Replacement, reduction, and refinement (the 3Rs) are employed to lessen the anticipated study harms.^26,27 Study benefit is stated and reviewed. However, unlike the applications of the 3Rs to reduce harm, there is no well accepted basis by which benefit can be reviewed and improved. We suggest in models proposed as bridges from preclinical into clinical (human) treatments that benefit can be increased by improved reproducibility and translatable study design.^28,29 In any preclinical animal model, the lack of reproducibility and/or translatability reduces the likelihood of benefit such that HBA is always biased toward harm.

CAN MORE BE DONE?

There have been calls to the research community at-large to adopt principles or rules in sharing preclinical data to counter the reproducibility crisis, though not necessarily the lack of translation.^13,30,31 However, there is little documented proof of adoption of “best practices” being consistently applied in areas such as randomization of animals, blinding of operators, use of power calculations, identification of primary and secondary endpoints before study start, and handling of data to include outliers. Several heuristic devices have been proposed to improve reproducibility, as tied to animal study design, documentation, and publication; these are discussed later.

While reproducibility is the ability to reproduce documented models and studies, translatability, as noted earlier, is a feature where the model replicates specific aspects of a human disease. For translatability of a model for human diseases or mechanism of disease, 3 target areas have been identified to fully define the translatability of an animal model. The first is “face validity,” which is alignment between the model and human phenotype, such as, for example, disease progression. The second, “construct validity,” is when the etiology of the phenotype is the same in humans and in the model. Lastly, “predictive validity” is whether the model predicts features of the human disease not presently known or understood.³² All 3 aspects should be documented as to what is known or not known about the intended model; for example, a model phenotypically similar to a human condition but with differing etiology may not be a translatable model. What is important for translatability is the knowledge of which aspects of the human disease are being replicated in the animal model, and for this article, where that discussion/debate occurs.

Ideally discussion, debate, and study redesign, when needed, should happen prior to IACUC submission. The authors recognize the challenge of this statement, noting IACUCs are neither composed for nor tasked with conducting rigorous scientific assessment of protocols. In the US system, for National Institutes of Health (NIH)-funded research, IACUC review and approval is required prior to going to study section. At the study section or other independent scientific review process, the study plan is often improved, changed, modified, or, in some cases, rejected at the time of review. The potential flaw in the system is easily visible. If the study is approved by an IACUC and funding is available, there is often no required step for additional scientific peer review process.

For grants receiving NIH funding, the expectation of the Office of Laboratory Animal Welfare is that “Peer review of the scientific and technical merit of an application is considered the purview of the NIH Scientific Review Groups (SRGs), which are composed of scientific experts from the extramural research community in a particular area of expertise”.³³ Though scientific peer review is accepted as a norm, its practice is highly variable. One example of standardization of a peer review process is the system used by pathologists supporting histopathology evaluation in regulatory safety studies.³⁴ However, there is no similar standardized process for assessment for the translational relevance of an animal study. Replication of the outcomes of an animal study in human test participants will be influenced by several factors, including the analytical rigor and reproducibility of the animal study, the biological appropriateness of the animal species and model, and the clinical relevance of the study. Improvement in study design lies in multidisciplinary review at both the project level and study level.^35–37

WHAT MAY BE DONE

For the IACUC to be a forum for influence and change in a proposed study, barriers for bioethical discussion beyond the 3Rs need to be recognized. Today, the expectation by regulators, high-level administrators, and inspecting and accreditation bodies can be defined as documentation of compliance. To demonstrate compliance, many articles on IACUC responsibility refer to the use of templates to ensure success in meeting inspecting and reporting agencies requirements.^38,39 Templates and standard assurance statements rarely have prompts to challenge study design or raise ethical discussion beyond the 3Rs and HBA.⁴⁰ The series of statements and/or questions to which the principle investigator (PI) must attest are tied to compliance of regulations and institutional policies.

The result is a deficit in guidance on discussion and scientific evaluation. Regulating authorities, such as the Food and Drug Administration⁴¹ and the Organization for Economic Co-operation and Development published principles for Good Laboratory Practices work to improve confidence in the reproducibility of animal study results and the reliability of study findings for decision-making.⁴²

More recently, there have been initiatives illuminating the gap in pharmacology or discovery studies. Mostly in Europe currently, there are organizations developing guidelines on planning and reporting studies. Examples include Design and Execution of Protocols for Animal Research and Treatment (DEPART) and Preparing research and Experimental procedures on Animals: Recommendation for Excellence (PREPARE).^43,44 While there is some overlap in the 2 tools, both use targeted questions and prompts to limit bias and improve validity. PREPARE attempts to address the needs of all stakeholders, while DEPART is for the PI and other scientists. The primary differentiation of PREPARE and DEPART from Animals in Research: Reporting in vivo Experiments is that the latter is a checklist of what is important to include in a publication on an animal study. PREPARE and DEPART are ideally employed before the study starts, whereas Animals in Research: Reporting in vivo Experiments occurs at the time of manuscript submission. Our guidance is to embrace 1 tool, for example, DEPART, as a scaffold on which to build the questions important for study reproducibility and translatability. The difficulty lies in determining where and when this discussion should happen. We noted for NIH grantees it would happen at the SRG; however, not all funding for studies are tied to NIH grants. One way forward is for each study unit, for example, a department of virology, to require an independent scientific peer review if a organizational does not come under the purview of the SRG. Independence is defined as a review involving a scientist with no vested interested in the study outcome. In short, prior to a study start, both ethical and independent scientific peer review must be completed. Only institutional policies and practice can close the gap for all studies.

As written, the primary focus of IACUCs is the welfare of the animal within the context of a study and the overall institutional program.⁴⁵ The questions and suggestions above would not be part of the typical IACUC discussion. In fact, we may need to ask ourselves if the present method of review and debate at IACUCs may inadvertently lower reproducibility and translatability by targeting discussion to HBA and the 3Rs.

Most IACUC handbooks and policies outline the requirements to demonstrate adherence to IACUC standards, including, but not limited to, the process for documentation of compliance, non-compliance, and any appeal process the institution may require.⁴⁶ Ethical debates are limited to discussion of potential harm to animal welfare and use of the 3Rs for modifying that harm, with presumption of benefit. Again, animal welfare as tied to reduction of harm is the primary focus. Bioethical principles are only now being proposed and discussed to add more guidance in the discussion on the need for an animal study.

At this time, new principles for bioethical review are defined and proposed. Recent publication of new bioethical principles is being discussed and debated by the community at large.^47,48 While it is too early to judge how the 6 proposed principles may be operationalized in IACUC discussions, they may move IACUC discussions beyond HBA and the 3Rs, and the reader is referred to their referenced book. It should be noted that these principles are not without controversy.

The Beauchamp/DeGrazia principles involve the 2 main challenges of translatable research: (1) Is there a social benefit for the study? and (2) What is the animal welfare cost? Under the first social benefit principle of “no alternative method,” we are asked to consider if the predicted benefits can be achieved only through animal studies. The difference from “replacement” is in the fact that we must seek if there are acceptable or qualified ways to get results not requiring animals, even if there is precedent for specific models. The second principle is one of “expected net benefit.” Here we must alter our present way of thinking on HBA and move to a risk–benefit analysis, which is about predicted risks and benefits for humans of the proposed research. The final principle under social benefit is about “sufficient value to justify harm.” Here too we must modify our consideration of HBA. This principle states that if we accept animals have moral status, even with all moral status not being equal, we must discuss and debate whether the study is of enough value to justify the harm caused to an animal.

The 3 principles of animal welfare can be summarized as follows. First is the principle of “no unnecessary harm.” This principle requires consideration of harm beyond the study design. Insults to animal welfare such as transit times, number of transfers, and other aspects of shipping routes are part of the debate. The second principle is “basic needs.” While we all comply with laws, they may not always meet the basic needs of a specific species. The example most of us are familiar with is housing paradigms for non-human primates. For example, the legal size does not require vertical space needed by some non-human primate species. And the third principle is “upper limits to harm.” Here, independent of the potential social benefit, a harm defined as too great should not be approved. Of course, there is currently much ambiguity as to what constitutes too great a harm, and the authors predict much debate and discussion if and when there is agreement on these 6 principles. The 3 principles for social benefit and 3 principles for animal welfare must be met to satisfy the proposed moral justification of a study.

CAN WE SUCCEED IN IMPROVEMENT OF STUDY DESIGN?

There is no easy solution to the challenge of increasing reproducibility and translatability. Consistent application of robust scientific design, power, blinding, and randomization is needed to ensure reproducibility and understanding the validity of the model for translatability. The IACUC is not responsible for scientific review, but it should be acknowledged that many studies do not get the benefit of the type of in-depth peer scientific review that takes place in NIH SRGs. It is also important to recognize there are no standards or guidelines for scientific review for the animal research component. There is often an overlap of science and animal welfare that IACUCs must discuss when they review protocols.²⁸ IACUCs would greatly benefit from the availability of animal model best practices for commonly utilized preclinical disease models.^49,50 There are many hundreds of types of animal models utilized and many differing constructs in the models used. Variations and differences in animal model study design and use are often due to the nature of the scientific questions being addressed. However, there is an unrealized opportunity for the biomedical community to share data by developing additional reviews, guidance, and best practices for commonly utilized preclinical models. Having best practice guidance would provide IACUCs with valuable resources to aid in the efficient review of protocols and would also provide many PIs useful tools for the optimization of commonly utilized animal models.

A community approach to the development of consensus best practice guidance on common preclinical models would enhance the 3Rs as well as the bioethical review of animal studies. Guidelines could help ensure maximal translational value and contribute to reproducibility. Best practice guidance of common preclinical models would certainly spare the biomedical community many unnecessary review hours by IACUCs. Many IACUCs struggle with identical issues concerning difficult models. A recent perusal of PubMed of the mouse dextran sodium sulfate (DDS) induced colitis model, a common model that leads to pain and distress in mice, revealed over 3500 papers. One can only imagine the number of hours spent by the many IACUCs that reviewed those thousands of study protocols, probably wrestling over similar difficult animal care and use issues. Using this model as an exemplar, reproducibility would be ensured if the study was powered correctly, there was true randomization of animals prior to study start, and the technical personnel involved with the study were blinded as to which were the control or treated animals. For translatability, we may agree that face validity is present; the phenotype is similar to what may be seen in some humans. However, there is no construct validity and the predictive validity is controversial.

The development of consensus approaches to model endpoints, experiment designs, and best practices for common models has been utilized for several years and there are many useful examples.^51–53 Nonetheless, there is still a gap in many of our platform models, such as the above referenced model: the need to be qualified through systematic, published reviews. There has been some work on these reviews, but much more is required.

CONCLUSION

We have a challenge to meet high bioethical, reproducibility, and translational standards; these are expected and accepted in our efforts to ensure judicious and respectful use of animals. Our current approach to meeting those expectations is largely defined by how to minimize harm through the application of the 3Rs. Animals as surrogates for humans is well established in today’s research paradigm, has significantly contributed to great progress in medicine and public health, and largely defines the “benefit” component of an animal study HBA. Animal studies to support human participant experimentation brings a different set of ethical expectations, which relate to our commitment to balance risk to a human test participant with the likelihood of receiving a benefit, where any potential benefit is defined by the preceding animal research. Current evidence suggests weaknesses in reproducibility and challenges in replicating animal study outcomes in humans. Though IACUCs and IRBs have the primary responsibility of ensuring ethical animal and human research, respectively, neither is positioned nor populated to assess the translational relevance of the animal studies. The field of translational biomedical research would significantly benefit from the development of standards and best practices for translational animal research. These best practices should include a rigorous scientific peer review process that precedes the ethical review. As suggested, already published heuristic tools such as PREPARE and DEPART provide guidance on increasing rigor with modest additional time requirements. The consequence of improving translational relevance of animal studies is supported by potential benefits across several value chains, including economic (eg, avoiding unnecessary animal and human clinical research), human health (improving the rate of success of human clinical trials supported by better animal evidence), and animal welfare (eg, strengthening the “benefit” side of the harm/benefit ratio). Of equal or even greater importance is an ethical value and expectation that the animal research we do in support of human health is relevant and translatable. Newer ways of probing ethical value and justification are captured by be the Beauchamp/DeGrazia principles. Our obligation in recognizing the moral status of animals and our promise to society of beneficial treatments is to ensure preclinical studies are designed to be both reproducible and translatable. In this way, the commitment to science and animals is met.

Potential conflicts of interest. None of the 3 authors have a conflict of interest on this topic.

References