Why Expert Experience is Vital to Medical AI

Having subject matter experts as part of the data annotation process is critical to Medical AI.

Firstly, the stakes are monumental. Medical AI systems aren’t making trivial choices; they’re guiding life-altering decisions, from diagnosis to treatments.

The margin for error? Non-existent. Even tiny discrepancies can have substantial impacts when applied to real-life medical scenarios.

Ultimately, then, building healthcare AI demands more than data science expertise. It demands nuance, context, and clinical insights.

Let’s unpack exactly why expertise matters most in the field of AI medicine.

Understanding Clinical Reality

When developing AI systems for healthcare, involving medical expertise from the outset prevents fundamental mistakes.

Medical experts bring several critical perspectives to AI development:

• Understanding which data points actually matter clinically.
• Recognising how medical practice varies across different settings.
• Identifying and analysing edge cases (uncommon, extreme, or rare features or events in the data) and managing their role in AI development.
• Validating that data labels match real clinical decision-making.

Without this expertise embedded in the development process, even technically impressive systems can fail in real clinical settings. 

Google Flu Trends is a key example. Despite having brilliant engineers and massive amounts of data, the project failed largely because it lacked input from epidemiologists who could have pointed out that flu patterns don’t always follow predictable seasonal trends. 

When the 2009 H1N1 pandemic hit during the summer months, the system missed it completely. It over-relied on seasonal flu trends, plotting a rather simple sin-esque curve, which you can observe below. 

Above: Google Flu trends ran from 2008 to 2015 and were abandoned for poor accuracy, partly because it lacked domain input from epidemiologists. Source: UCL.

Conversely, BlueDot, founded by a frontline physician who had worked through the SARS epidemic, successfully identified early signs of COVID-19 nine days before WHO’s first statement. 

Their team combines medical expertise with technical skills – their CTO has spent his career working on medical software, and they maintain a comprehensive team of medical professionals to guide their work.

This illustrates how medical AI doesn’t revolve around technical processes and data. Reliability is factored in via the human experts involved in the project.

Addressing Ethical Concerns

When healthcare professionals are excluded from AI development, the results can be ethically disastrous in ways that aren’t immediately obvious. Bias is not purely a matter of ethics – it can manifest in real-world patient outcomes, often with harmful consequences. 

For example, a model trained on biased or incomplete data could misdiagnose illnesses, prescribe ineffective treatments, or overlook vulnerable populations entirely.

There are several ways AI systems can introduce bias into medical decision-making:

• AI systems can “hallucinate,” presenting seemingly reasonable but factually incorrect information
• Clinicians may over-rely on AI instead of their own medical judgment
• Technical teams often optimise for statistical performance without understanding real clinical implications

One of the most telling examples comes from dermatology, where AI promised to revolutionise skin cancer detection and diagnosis. 

However, a fundamental problem emerged: many of these AI systems performed poorly when examining darker skin tones.

A comprehensive 2024 review in the Journal of Investigative Dermatology revealed why – the systems had been trained primarily on images of light skin, creating a dangerous blind spot that any dermatologist would have recognised immediately.

This reflects a broader pattern in medical AI. Recent studies have shown that AI systems can actually detect a patient’s race from medical images like chest X-rays – something not normally detectable by human doctors. 

When combined with evidence that marginalised groups often receive inferior care, this raises serious concerns about AI’s potential to amplify existing healthcare disparities.

“An inherent risk with learning models is if there are biases in the training data — especially as it relates to underrepresented population groups. Some examples include race, nationality, and socioeconomic background. Training data transparency and explainability will be critical to build confidence that the model is taking into consideration the many variables that go into patient care.” – Tony Lee, JD, Chief Technology Officer with Hyperscience, a New York AI company. Source: Wiggin.

How Do Experts Address Medical Bias?

Addressing bias in medical AI starts with examining systemic flaws in data and design. 

Experts identify gaps in representation – ensuring diverse populations, conditions, and contexts are adequately captured. They work closely with clinicians to contextualise data and identify details like how symptoms manifest differently across demographics. 

Rigorous stress testing evaluates whether systems perform equitably for all groups, not just averages. 

Experts also push for transparency, creating models that are interpretable and accountable. Combating bias is a continual process that prioritises fairness, accuracy, and patient trust.

Further reading: Four ways AI is transforming healthcare today.

Bridging the Gap Between AI and Practice

Making AI work in healthcare means understanding the complex realities of clinical practice. Even the most sophisticated AI system fails if it doesn’t mesh with how healthcare actually happens on the ground. 

A striking example, as highlighted by the Ada Lovelace Institute, comes from a study of Sepsis Watch, an AI “early-warning system” designed to help hospital clinicians diagnose and treat sepsis. 

While the system was technically sound, it only succeeded after close engagement with nurses and hospital staff to ensure it triggered alarms in an appropriate way and led to meaningful responses. 

The key discovery? Clinicians needed to take on an intermediary role between the AI system and other clinicians to make the tool work effectively in hospital settings.

Medical experts help bridge this gap in several critical ways:

• Aligning AI systems with existing clinical workflows
• Ensuring AI outputs translate into meaningful clinical actions
• Identifying real-world implementation challenges early
• Helping design practical interfaces and alerts
• Supporting effective training and integration

The bottom line is, clinicians can’t be only end users of AI – they need to be actively involved in influencing how these systems work in practice. 

At Aya Data, we know that bridging the gap between AI capability and clinical practice means creating tools that work in the real world of healthcare, not just in controlled testing environments.

The Human Element in AI Training

Data is the foundation of all AI systems. Medical data involves highly complex information that requires deep clinical understanding to label properly.

Those labels – also called data annotations – instruct algorithms on understanding and interacting with the data to ultimately drive analysis and decision-making. 

In essence, while AI can process vast amounts of medical images or patient records, it takes experienced healthcare professionals to identify what those data points actually mean.

Consider the process of labelling medical images. A technical team might mark basic visual features, but a radiologist knows that a slight shadow could be meaningless in one context and critical in another. 

Medical domain experts understand which variations matter clinically and which don’t. This expertise shapes not just what gets labelled but how those labels reflect real medical decision-making, involving:

• Identifying subtle clinical indicators that non-experts might overlook
• Understanding how symptoms and conditions present differently across patient populations
• Recognising rare but important cases that need special attention
• Validating whether AI interpretations align with clinical reality
• Providing context about how findings would influence actual treatment decisions

At each stage, healthcare professionals help refine the system’s understanding, correct misinterpretations and ensure the AI remains grounded in clinical reality.

This human-in-the-loop strategy ensures AI systems learn from genuine medical expertise, not just simple pattern matching. 

At Aya Data, we’ve built a network of medical specialists across different disciplines to enable this deep level of expertise in our AI development. 

Our subject matter experts integrate their skills and expertise into our entire AI development pipeline. 

From radiologists and pathologists to clinical specialists, our network grounds training data in real medical understanding. 

Building Trust Among Stakeholders

Healthcare AI exists within a complex web of relationships between doctors, patients, administrators, and regulators. 

Building trust among these stakeholders demands a true, authentic comprehension of healthcare’s human elements.

Medical experts help build trust in several ways:

• Understanding clinician concerns about workflow disruption and patient safety
• Identifying potential resistance points in implementation
• Ensuring systems complement rather than replace clinical judgment
• Translating technical capabilities into meaningful clinical benefits
• Building bridges between technical teams and healthcare staff

When medical experts guide AI development, they help ensure these systems work in harmony with clinical practice rather than against it. 

They understand what makes doctors hesitant to adopt new technologies, what makes patients anxious about AI in their care, and what regulators need to see to be confident in patient safety.

Experts also recognise that trust must be earned gradually. Systems need to demonstrate reliability over time, transparency in their limitations, and clear benefits to patient care. 

Summing Up

Evidently, the role of medical experts in healthcare AI is fundamental to success.

From ensuring data quality and addressing ethical concerns to bridging the gap with clinical practice and guiding proper validation, expert involvement shapes every aspect of effective healthcare AI.

At Aya Data, we put this understanding into practice through our strong network of subject matter experts. 

Our partners work alongside our technical teams throughout the entire project process – from data collection and annotation through to building and deploying AI-powered models for medical institutions. 

This ensures our solutions aren’t just technically sophisticated but truly ready for the complexities of real healthcare delivery.

Contact us today to kickstart your medical AI project.