← Blog한국어

Evaluating Skin with AI

Skin severity rarely has a single right answer. This is the question a dermatologist carried through a university hospital and a medical AI company, 'how much can we trust a doctor's judgment?', and the story of solving it with AI.

Soo Ick ChoSoo Ick Cho
  • skin evaluation
  • medical AI
  • INSKINVIEW
Evaluating Skin with AI

In my last post, I wrote that measuring skin turns out to be far shakier than it sounds. That naturally raises a question: "If the devices can't be trusted, why not just have a trained doctor look and judge?" I believed that for a long time, too. But doing AI research, I ran into something uncomfortable; even a doctor's judgment doesn't settle into a single right answer.

My first paper was about telling skin cancer apart on the lips

At the university hospital, my first research topic was using AI to identify a rare skin cancer that appears on the lips. Back then, AI research on diagnosing disease was just emerging, and because the model could distinguish even an uncommon cancer, the work was accepted into a major dermatology journal (British Journal of Dermatology).

Diagnosis was, relatively speaking, the easier problem; because "cancer or not" is treated as having a right answer. (Strictly speaking, even diagnosis is closer to a probability.) But that kind of diagnostic research was already saturating quickly, so I turned to the next question: severity grading.

"Severity" didn't have a single right answer

I went on to study AI grading of severity in acne and atopic dermatitis. Both were recognized as novel and published in strong journals, but the real wall I hit was something else.

To train an AI, you need "ground truth." So I showed the same photos to several dermatologists and had them grade severity, and their answers diverged. Most overlapped, and wildly different calls were rare (say, on a 4-point scale, one grades it a 3 and another a 2), yet a single correct answer simply didn't exist.

Unlike "cancer or not," severity is a problem where several experts' judgments scatter into a distribution. It needed a different approach than diagnosis, and my research was about solving exactly that.

This wasn't a dermatology-only problem

Later, building pathology-analysis AI at a medical AI company (Lunit), I hit the same wall. We trained on the labels pathologists marked on tissue images. And even those labels differed from one pathologist to the next. Looking at the same tissue, one might call it a premalignant lesion and another a malignant one.

That's when it clicked: the problem I'd met in dermatology wasn't unique to dermatology. It runs through medicine broadly. And through both dermatology AI and pathology AI, I learned firsthand how to handle this "scattered ground truth" with AI.

So the real question is how we evaluate

The key isn't to hand the AI one rigid "correct answer," but to let it learn the distribution of judgments that actually exists in clinical practice, the probability, in a sense. Rather than erasing one expert as "wrong," you take the full spread of several experts' judgments as the training material itself. I developed AI models this way at Seoul National University Hospital and at Lunit, and confirmed that this approach fits severity assessment better. That know-how is what I'm now applying to INSKINVIEW's AI models.

References

Studies referenced in this post.

Dermatology AI

Pathology AI