The Risks and Challenges of Artificial Intelligence in Endocrinology Free

Abstract

Artificial intelligence (AI) holds the promise of addressing many of the numerous challenges healthcare faces, which include a growing burden of illness, an increase in chronic health conditions and disabilities due to aging and epidemiological changes, higher demand for health services, overworked and burned-out clinicians, greater societal expectations, and rising health expenditures.

While technological advancements in processing power, memory, storage, and the abundance of data have empowered computers to handle increasingly complex tasks with remarkable success, AI introduces a variety of meaningful risks and challenges. Among these are issues related to accuracy and reliability, bias and equity, errors and accountability, transparency, misuse, and privacy of data.

As AI systems continue to rapidly integrate into healthcare settings, it is crucial to recognize the inherent risks they bring. These risks demand careful consideration to ensure the responsible and safe deployment of AI in healthcare.

Background

Artificial intelligence (AI) often generates an uneasy mix of awe and fear. The excitement surrounding AI in healthcare can lead to a sensationalized view of its capabilities, eclipsing the range of technological and operational challenges as well as the safety and ethical concern it introduces (1, 2). AI encompasses a diverse range of sophisticated computer-based analytical tools developed to simulate human intelligence. The foundation of modern AI lies in large language models, which exhibit the ability to piece together a rudimentary understanding of how our world functions and communicate in a compelling manner. These models can have a staggering number of parameters, surpassing a trillion or more, representing a scale of processing beyond human comprehension (3).

Machine learning, a dominant subfield within AI, focuses on enabling computers to learn from data (3). Compared to traditional simple inferential statistics, machine learning employs sophisticated nonlinear learning methods. Among these algorithms, artificial multilayered neural networks, often referred to as deep learning, have emerged as particularly successful, drawing inspiration from the connections in the human brain (3).

Together, AI and machine learning offer a remarkable range of opportunities and efficiencies to medicine and endocrinology (Table 1). Balancing the risks with these potential benefits is already creating meaningful challenges for the profession and our community.

Ethics

Ethics has suffused the medical profession since its inception and must be applied to AI. While AI can revolutionize healthcare, it lacks common sense and an ethical framework, making human guidance and oversight crucial in data generation, transformation, and relationship determination. It is clear that human values, human rights, human dignity, and human freedoms are each expected to inform the governance of AI (4). General principles that have been surfaced as ethic prerequisites include human agency and oversight (obedience of the AI to humans), beneficence (to broad populations now and in the future), nonmaleficence and virtue, privacy and data governance, transparency of the data, diversity, and nondiscrimination and fairness (5). While many scholars agree on these principles, they disagree on their interpretation and how to resolve competing principles (4). Journals, including the Journal of Clinical Endocrinology and Metabolism, are a case in point, having recently strengthened their author guidelines to remind authors they are required to take full responsibility for their submission “regardless of the use of any AI” in their drafting: an equipoise that facilitates careful and transparent use of a new technology (6).

Accuracy and Reliability

AI offers vast potential for improving patient outcomes through advances in population health management, risk identification and stratification, diagnosis, and treatment (2). AI has already demonstrated the capacity to interpret data in ways that are comparable to the performance of medical professionals in variety of forms, such as understanding multifaceted medical cases and analyzing radiologic images or pathology sections (7, 8). Additionally, machine learning can estimate risks and guide decision-making, such as in guiding the titration of testosterone injections or creating reminders for scheduled dose increments (9). Larger health system issues, such as effective provision of public health or healthcare, are also a lattice of information processing tasks. Machine learning has the potential to improve hypothesis generation and hypothesis testing tasks within a health system by revealing previously hidden trends in data (Table 1), and thus could make substantial impact both at the individual patient and system level (10). However, AI faces challenges like overfitting in small imbalanced datasets and biases in healthcare delivery and outcomes, leading to misleading predictions in underrepresented demographics.

AI addresses data and analytics in healthcare but fails to address behavioral changes in patients and physicians. Thus, algorithmic recommendations must be rigorously tested for their impact on clinical outcomes before widespread adoption, even with innovative models (2).

Autonomy and Accountability

The rapid pace of AI advancements in healthcare raises the issue of professional responsibility and accountability (1). Since physicians remain ultimately responsible for patient care decisions, they cannot relinquish accountability to AI. The first step in creating accountability is to help physicians and healthcare professionals to learn how these systems work and how to use them most effectively; such training should help demonstrate the collaborative value of an AI tool and assuage natural fears that the technology will replace their expertise (11). Clinicians who use AI for patient care or academics must also be held accountable for disclosing its use to the patient, to the payor, or to their colleagues.

One significant concern is the potential for AI to undermine the traditional role of doctors, if AI models gain and undermine the trust that the public places in medical professionals and their professionalism. Though rarely acknowledged, doctors and healthcare professionals make many mistakes (12). Regulators and users can never expect perfection from a computer model, but what error rate for a diagnostic algorithm using AI would be acceptable? The same rate as an experienced doctor? A “typical” doctor in that field?

Learning from mistakes is crucial for growth, particularly in cognitively demanding jobs such as in healthcare. Mistakes, whether careless, systematic, or misconceptions, when corrected, can be highly instructive provided they are reflected on and learned from (13). AI may well reduce errors in clinical decision-making, which is important and valuable, but may also reduce the opportunity for reflection on prevented errors and interfere with the ability to accrue associated wisdom.

AI models themselves must also be held accountable to humans. The otherwise impenetrable machinations of large language models may need to facilitate an explanatory component that works to explain to a user the rationale and values used to derive a recommendation or action. Such explanations are likely to be required by regulators. Consequently, there must be ways to force these models to control their own learning and unlearn when needed.

Misuse

The possibility that AI can be used to manipulate individual or group behavior calls for policies that ensure that the AI remains subordinate to user control and provides transparency. While misuse is already apparent in medicine, some of the most notable examples have related to targeted commercialization and recruitment (14).

To reduce the risk of intentional or inadvertent misuse, medical professionals must be adequately trained to understand the limitations, risks, and potential pitfalls of AI technologies they employ. As AI algorithms gain proficiency in diagnosing diseases and suggesting treatment plans, there is an increased risk of doctors becoming overly reliant on, having overconfidence in, and misusing these systems. These patterns of behavior would lead to reduced critical thinking and decision-making capabilities, inducing errors, negligence, intellectual laziness, and complacency (15).

While there is an obvious need for training of clinicians to facilitate effective use of AI, prevent misuse, and generate accountability, the education infrastructure itself can be a target for manipulation. AI offers a variety of benefits and opportunities to use technology to identify and target gaps in awareness of skill with precise training interventions and to leverage rehearsal, repetition, and reminders that promote memory consolidation and skill maintenance (11). Without accountability and transparency, AI could be used to manipulate information flows to clinicians (just as it has successfully been used in commercial sales), and result in promulgation of misinformation and ultimately loss of trust (14).

Patients may also misuse the information gleaned from AI tools and must be told how their information is and will be used or reused, and where care recommendations or information came from.

Avoiding AI misuse will require involvement of clinicians and citizens in AI development, training in the safe and effective use of these tools among healthcare professionals, facilitating awareness and literacy among patients and the general public, and facilitating explanation and direction from national authorities and guidelines to clinicians before there is a proliferation of easily accessible online and mobile AI solutions.

Privacy

Privacy in healthcare is an expectation and a right. With the digitalization of health information, healthcare organizations and providers have faced growing challenges with securing increasing amounts of sensitive and confidential information while adhering to federal and state privacy and security regulations (5). AI systems heavily rely on vast amounts of patient data to train and improve their accuracy. As such, patient medical records, including sensitive information, are frequently stored and processed by AI algorithms. A major challenge arises in safeguarding this data from potential breaches and unauthorized access (16). Data breaches could lead to identity theft, misdiagnoses, or manipulation of medical records. The multiple datasets available in healthcare systems create the opportunity for bad actors to identify potential targets and victims with AI. Ensuring robust cybersecurity measures and adhering to stringent data protection regulations are imperative to mitigate these risks and maintain patient trust in AI-driven healthcare technologies (16).

AI technology intentionally collects human data from its users, and those users often do not realize their data is being used to inform and drive large language models. Although many countries require user consent before information is shared with AI, most users do not know whether, how, and when their data is used.

Equity

One of the most critical concerns regarding AI adoption in healthcare is algorithmic bias (17). Bias can occur for various reasons; for example, the data used to train AI applications—as well as the rules used to build algorithms—might be biased. If historical healthcare data contains inherent biases based on factors such as race, gender, or socioeconomic status, AI systems can perpetuate and even amplify these biases when making medical recommendations or decisions (5, 18). For example, if access has traditionally hindered the ability of immigrants to afford an expensive therapeutic, an AI model could perpetuate that bias by recommending an agent that is most often used among immigrants, rather than the most effective or appropriate medication. Bias might occur because of a variance in the training data or environment and how the AI program or tool is applied in the real world. For doctors, relying on AI-generated recommendations without critically evaluating potential biases could result in unequal treatment and exacerbate existing health disparities (19). Addressing algorithmic bias demands ongoing research, rigorous data curation, and continuous oversight to ensure AI applications promote equitable healthcare outcomes for all patients (19).

Investment in AI and the resulting innovations will ultimately be expected to deliver value and profit to its developers. While these tools and services may seek and promise to deliver efficiencies at scale, it is not clear that the healthcare system can tolerate the rising costs of these technologies, or avoid the inevitable inequities that arise when cost is such a major factor in healthcare delivery.

Options to Manage the Threat and Challenges

With the flourishing range of options and solutions that AI offers patients, clinicians, and administrators in endocrinology (table) there can be little doubt that they will play an increasingly critical and central role in healthcare. To manage these challenges, it will be essential to engage a broad community to facilitate the growth and development of the most effective tools, enhance transparency into the operations of AI and its use in publishing, training, and patient recommendations, and transform training so clinicians in practice and the next generation of healthcare workers are able to navigate the complexities of these new technologies (20). From a policy perspective it will be important to extend AI regulatory frameworks so that they address healthcare-specific issues, set criteria for the assessment and testing of these systems including the evaluation of bias, and take steps to optimize equity in their deployment and availability. The pace of evolution of AI technology and its unpredictability has so far befuddled regulators (21). The unsuccessful efforts to create international alignment must be restarted so that the global promise of AI can also benefit from global oversight with shared standards (22).

Conclusion

Throughout the history of technology, there have been critical junctures where our excitement for a particular innovation surpasses our capacity to impartially assess its merits and shortcomings: we have arrived at that cliff edge and must now be deliberate about how we proceed before these technologies proliferate ahead of our profession's readiness to adapt and absorb. While AI has proven valuable for physicians, aiding in self-learning, uncovering complex relationships in large datasets, and providing automated clinical decision support, the range of risks and challenges is only now becoming clear.

Implementing AI in medicine requires careful consideration to maximize its scope and utility. It has the potential to unveil hidden insights and trends in data, benefiting individual patients and the entire healthcare system. The choice we face is whether to control or even halt the development of advanced AI, which some fear may threaten or replace clinicians, or to embrace experimentation with a technology that can shape this century's healthcare as evidence-based medicine did in the last. Guidelines for AI use in healthcare will help foster an open dialogue between doctors, patients, and AI developers that will be essential in ensuring ethical and responsible AI implementation in the medical field. Striking a delicate balance between leveraging AI's advantages and preserving the human touch in medicine is essential to ensure that patient care remains both effective and compassionate.

While technology offers opportunities to enhance care accuracy, safety, and efficiency, we must prioritize the ethical implications of AI's evolving power. It is crucial to establish an ethical framework that informs the regulatory and technical architecture that protects against AI autonomously integrating with critical systems, such as healthcare infrastructure. To ensure harmonious coexistence with AI in the long term, seamless collaboration between human operators and AI must be fostered, maintaining human control and oversight over the evolving remarkable technologic advances that AI promises.

Disclosure

The author has no conflicts of interest to disclose.

Data Availability

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

References

Abbreviation

 
• AI

  artificial intelligence