Study shows: Skin cancer apps lack information and labelling

AI-based apps for examining skin changes are popular. However, research shows that these apps need to improve their information.
An increasing number of mobile apps use AI to analyse images of suspicious skin lesions. However, many self-examination apps fail to provide adequate information about how they function. This is demonstrated by two studies led by Åsa Ingvar, a consultant dermatologist at Skåne University Hospital and an associate professor at Lund University.

Early detection of malignant melanoma, a form of skin cancer, is crucial. Today, many mobile apps promise quick and convenient analysis of self-taken images of moles and birthmarks. Yet, it is difficult for an individual consumer to assess whether these apps meet the necessary standards, notes Åsa Ingvar, who has conducted two studies on AI-based apps for self-examination of skin lesions.

“To determine if an app is reliable, users need information on how it is constructed – and this is often lacking today,” says Åsa Ingvar.

The studies were conducted in Australia, aiming to investigate the type of labelling and information provided by manufacturers regarding these apps, and to propose guidelines for how the information should be presented.

Åsa Ingvar, consultant dermatologist at Skåne University Hospital and an associate professor at Lund University.

Did not meet recommendations

In one of the studies, researchers reviewed 21 AI-based dermatology apps and found that none of them fully complied with standard labelling recommendations.

“This is a concern because users do not gain knowledge about the apps’ limitations,” says Åsa Ingvar.

For example, she mentions that several AI-based apps are often “trained” on images of skin lesions from individuals with light skin tones, which may make them less effective in analysing images from people with different skin colours. The apps may also be limited if they are only trained on images from a specific type of camera, or if they have been trained on too few or no images of certain skin lesions.

“It is well-known among developers that AI apps are sensitive to deviations from the data they were trained on. But this knowledge must also be disseminated among users, whether they are healthcare professionals or not,” says Åsa Ingvar.

Low threshold

Another limitation is that the apps often have a low threshold when it comes to warning users about skin changes.

“App developers do not want to risk missing something like malignant melanoma, so they would rather advise seeking medical care too often than too seldom. This can lead to an increased burden on traditional healthcare services, with a higher proportion of unnecessary visits that do not provide a medical benefit in terms of improving a health condition,” says Åsa Ingvar.

Expert panel assistance

In the second study, the researchers enlisted the help of an expert panel to clarify the requirements and labelling that should apply to AI-based dermatology apps. The expert panel concluded, among other things, that it should be clear how the apps were tested and trained, which clinical studies, if any, have been conducted, and if there are any limitations in the app.

“Like many others, I have high hopes for the use of AI in healthcare. However, AI-based apps have limitations when it comes to detecting malignant skin cancer. Hopefully, clearer guidelines will lead to improved quality,” says Åsa Ingvar.

Publications

Examining labelling guidelines for AI-based software as a medical device: A review and analysis of dermatology mobile applications in Australia - PubMed (nih.gov)

Minimum labelling requirements for dermatology artificial intelligence-based Software as Medical Device (SaMD): A consensus statement - PubMed (nih.gov)

Last updated: 2024-08-19