Encoded Inequity: Algorithmic Decision Tools in Clinical Medicine and the Perpetuation of Racial Health Disparities

When the Algorithm Inherits the Archive

The integration of artificial intelligence into clinical medicine has been accompanied by substantial optimism—promises of more precise diagnosis, more efficient resource allocation, and the elimination of idiosyncratic human judgment from high-stakes medical decisions. What has received comparatively less attention in mainstream health discourse, though considerably more in the peer-reviewed literature, is the degree to which these systems encode and operationalize the inequities already embedded in the historical data from which they learn. A clinical decision support algorithm trained on decades of electronic health records inherits not only the clinical patterns those records contain but also the biases, omissions, and structural distortions that shaped them.

This is not a hypothetical concern. A landmark 2019 study published in Science by Obermeyer and colleagues demonstrated that a widely deployed commercial algorithm used to identify patients for high-risk care management programs systematically underestimated the health needs of Black patients relative to white patients with equivalent illness burden. The algorithm used healthcare cost as a proxy for health need—a seemingly neutral technical choice that, when examined through an equity lens, reflected the reality that Black patients with equivalent illness severity had historically incurred lower costs, in part because of reduced access to care. The algorithm interpreted lower historical expenditure as lower need, effectively laundering structural underinvestment in Black health into a clinical recommendation.

Racialized Variables and the Architecture of Bias

Beyond the proxy variable problem, a growing body of research has documented the use of explicit racial adjustment factors in clinical algorithms that systematically alter diagnostic and treatment thresholds along racial lines. The most extensively studied example involves the estimated glomerular filtration rate (eGFR), a measure of kidney function used to determine eligibility for nephrology referral, transplant listing, and other interventions. For decades, a race coefficient was incorporated into the standard eGFR equation, producing higher estimated kidney function scores for Black patients than for non-Black patients with identical creatinine levels.

The clinical consequence was substantive: Black patients were systematically classified as having better kidney function than their biology indicated, delaying referral to nephrology care and, in some analyses, reducing access to kidney transplantation. A 2020 study in the Journal of the American Society of Nephrology estimated that removing the race coefficient would reclassify a meaningful proportion of Black patients into more severe chronic kidney disease categories warranting earlier intervention. Following sustained advocacy from nephrologists and health equity researchers, the National Kidney Foundation and the American Society of Nephrology issued recommendations in 2021 to eliminate the race variable from eGFR calculations—a significant, if overdue, corrective.

Similar race-based adjustments have been identified in algorithms governing pulmonary function testing, vaginal birth after cesarean (VBAC) risk scoring, and cardiac risk stratification. In each case, the incorporation of race as a biological rather than social variable naturalizes disparities that are in fact products of differential exposure, discrimination, and structural disadvantage.

Feedback Loops and the Compounding of Inequity

From a public health systems perspective, one of the most concerning properties of biased clinical algorithms is their capacity to generate self-reinforcing feedback loops. When an algorithm trained on biased historical data produces recommendations that result in differential care allocation, the subsequent clinical data generated by those recommendations becomes part of the training corpus for future model iterations. Disparities are not merely preserved; they are amplified and institutionalized within the computational infrastructure of the health system.

This dynamic is particularly acute in the context of predictive risk stratification tools used to allocate care management resources, hospital beds, or specialist referrals. Research published in Nature Medicine and the New England Journal of Medicine has documented differential algorithm performance across racial and ethnic groups in applications including sepsis prediction, deterioration early warning systems, and readmission risk scoring. In settings where algorithmic outputs directly govern resource allocation decisions, underperformance for minority patient populations translates into measurable differences in care intensity and, potentially, in outcomes.

The public health implications extend beyond individual clinical encounters. At the population level, systematic underallocation of preventive and chronic disease management resources to Black and Latino patients—mediated in part by biased algorithmic tools—contributes to the persistent disparities in cardiovascular disease, diabetes, and chronic kidney disease mortality that public health surveillance systems continue to document.

Accountability Gaps in Algorithmic Governance

Despite the accumulating evidence base, the regulatory and institutional framework governing clinical algorithm deployment in the United States remains underdeveloped relative to the scale and speed of adoption. The Food and Drug Administration has expanded its oversight of software as a medical device, including certain AI-based clinical decision support tools, but the regulatory pathway for many embedded algorithmic applications remains ambiguous. Health systems and electronic health record vendors frequently deploy third-party algorithms without independent validation of performance across demographic subgroups, and algorithmic audit requirements are not yet standardized across accreditation or regulatory frameworks.

The Office of the National Coordinator for Health Information Technology and the Department of Health and Human Services have issued guidance documents addressing algorithmic bias, and the Biden administration's 2023 Executive Order on artificial intelligence included provisions relevant to health applications. However, translating policy frameworks into enforceable standards with meaningful consequences for non-compliance remains an ongoing challenge.

Several academic medical centers and health systems have begun developing internal algorithmic equity review processes, drawing on frameworks proposed by researchers at institutions including Stanford, Harvard, and the University of Pittsburgh. These approaches typically involve systematic disaggregation of algorithm performance metrics by race, ethnicity, sex, and socioeconomic indicators prior to deployment, combined with ongoing post-deployment monitoring. While promising, such initiatives remain voluntary and inconsistently implemented.

Toward Equitable Computational Medicine

The integration of machine learning into clinical practice is not inherently inequitable, but the conditions under which it has been pursued in the United States have frequently reproduced and entrenched existing disparities. Correcting this trajectory requires action across several domains simultaneously: rigorous pre-deployment equity auditing as a condition of health system adoption; mandatory disaggregated performance reporting; elimination of unjustified race-based biological adjustments; and meaningful inclusion of communities most affected by algorithmic harm in algorithm development and governance processes.

Public health researchers have a critical role to play in this effort—both in generating the epidemiological evidence necessary to characterize the population-level consequences of algorithmic bias and in advocating for regulatory and institutional frameworks adequate to the challenge. Precision medicine that systematically imprecises its estimates for marginalized populations is not precision medicine. It is a technologically sophisticated iteration of a much older problem.