Health & AIClinical AIAfroTech 2026November 2026
Same Labs, Different Answer: Waverly Rose Brim on Where Harm Enters Clinical AI
At AfroTech 2026, AI research engineer Waverly Rose Brim walked a room through one kidney test, from blood draw to AI model, and showed how a model can score 90 percent and still not be fit to ship.
HOUSTON Two patients. Blood drawn the same morning. Same age, same sex, same lab panel. Waverly Rose Brim asked the room whether their kidney function estimates should be different.
For more than 20 years, she said, they were. The standard kidney function equation returned a higher number for the patient recorded as Black, and a higher number reads as healthier kidneys.
Brim is an AI research engineer who builds clinical AI systems, from surgical computer vision to tools that test medical prediction algorithms for safety and fairness. She has authored more than 20 peer-reviewed publications in medical AI and neuro-oncology, previously moderated the “Dr. AI” session at the World Economic Forum’s Annual Meeting of the New Champions, and is pursuing an M.S. in Artificial Intelligence at Johns Hopkins University. Her hands-on workshop at AfroTech's HealthStack, "Building Fairness into the Data Supply Chain," put the room inside live coding notebooks built on real CDC data and asked them to find where harm enters a clinical AI pipeline. Her answer: rarely in one place, and often long before the model.
Work someone has to trust with their life
She opened with a photograph of Private Casberry B. Freeman Sr., a Tuskegee Airmen engineer she calls Great Uncle Buddy. Brim is his great-grandniece. The pilots flew the missions, she said, but they trusted their lives to the ground engineers who kept the planes ready.
"When I look at this photograph, I think about what it means to do work that someone else must trust with their life."
Clinical AI, in her framing, runs on the same partnership. Clinicians bring the judgment to read a result in the context of a patient's life. Engineers bring the expertise to examine how that result was produced. Left unexamined, she said, biased models are set to widen racial health disparities. Handled with care, AI could become a great equalizer.
A number becomes a decision
The case study was chronic kidney disease, which Brim noted falls hardest on Black Americans, who are more than three times as likely as white Americans to develop kidney failure. Doctors rarely measure kidney function directly. They estimate it. A blood test returns creatinine, an equation turns it into a single number called eGFR, and that number sets the stage of disease, which in turn shapes referrals, medication eligibility and transplant lists.
Her example was a 50-year-old Black man. Under the older equation his number was 62. Under the newer one it was 56. A small change, until you notice the line at 60. At 62 he sits on one side of a clinical threshold; at 56 he sits on the other.
Taking race out widened the gap
In 2021, a national task force recommended a new equation without race in it. Brim asked the room what happened next. The intuitive answer is that the accuracy gap closed. It didn't. With race in the equation, kidney function in Black patients was overestimated by about 3.2 units relative to other patients. With race removed, the gap flipped direction and more than doubled, to 7.6.
"Removing the coefficient didn't close the gap. It widened it."
Adding a second blood marker, cystatin C, shrank the gap to about 3, but not to zero. The problem, she argued, runs deeper than one variable. And changing a number doesn't automatically change care. Research she cited found that after the switch, many Black patients moved into a more severe disease stage on paper, but kidney specialist referrals showed no significant change. What moved transplant outcomes was a deliberate 2023 policy that restored lost waitlist time to Black candidates.
"Changing the equation wasn't enough here. Changing a policy was."
Ninety percent, and it still doesn't ship
Then the room built a model. Most clinical AI teams, Brim said, simply leave race out of the inputs and move on. But the eGFR value they keep was already calculated using race, so the model inherits it anyway. Measured by AUROC, the headline metric engineers love, swapping the equations changed almost nothing. The model scored about 90 percent either way.
"The AUROC answered the question that we asked. The problem is that we asked a question that could not see harm."
Split the same predictions by insurance status, and the picture changed. The model performed at about 91 percent for insured patients and about 78 percent for uninsured ones. Because most patients in the test set were insured, the strong overall score hid the weak one underneath it.
"If you're not disaggregating and you're not looking at it, you're not going to see it."
Brim had set her rule before looking at the results: no group could fall more than a set margin behind the rest. The uninsured group did. Her verdict was simple: "This mortality model, it's not going to ship." Setting the rule first, she told the room, is the skill to take home, because "otherwise it's too easy to justify the outcome after you have already seen the numbers."
Who decides what to look for
Brim chose to check insurance. Nothing in the pipeline told her to. That, she said, is the point: an audit only finds the gaps someone thinks to name, which is why it matters who is in the room.
"We have to ask the questions that are relevant to our community."
Small percentages, she reminded the room, are still people. In her national estimate, the equation change alone moved roughly 427,000 Black adults across the kidney disease line. Finding them takes work, and she was clear about who she expects to do it: the people who care enough to look.