An Approach to Improve the Efficiency of Deep Learning Framework for Skin Cancer Classification Utilizing Dermoscopy Images

Abstract

Accurate identification and classification of skin cancer play an important role in early diagnosis, which is vital for reducing the mortality rate worldwide. However, challenges such as the variation in the appearances of skin lesions, class imbalance and limited computing resources make it hard to develop reliable models. Consequently, in this study, we propose a novel deep learning framework for skin cancer classification combined with self-supervised learning (SSL), multi-axis attention mechanisms and knowledge distillation to enhance the accuracy and efficiency. In order to overcome the dataset imbalance, medicalaware augmentation and dual-level class balancing methods such as focal loss and class weighting, were applied during training. Based on SSL, our framework utilizes MoCo-v3 for robust feature extraction from unlabeled data that helps the teacher model using a Vision Transformer (ViT) backbone enhanced with Low Rank Adaptation (LoRA) and multi-axis attention mechanism to identify the complex pattern of skin lesions. Knowledge distillation transfers the knowledge from the teacher model to a lightweight student model based on TinyViT with custom modifications that achieve 94.14% accuracy on the PAD-UFES-20 dataset in an efficient manner using only 5 million parameters. Evaluated on benchmark datasets including HAM10000, ISIC-2019 and Pad-UFES-20, the teacher model with an accuracy of 93.72%, 92.86% and 94.56%, respectively, outperformed both ensemble and pre-trained baseline models such as ConvNeXt, ResNet101, DenseNet201, EfficientNetB4, TinyVit, EfficientNetB1, MobileNetV3, ResNet18, DenseNet121 and ViT with a significant improvement in accuracy, precision, recall and F1-score. The student model ensured similar performance with great efficiency and this made our model suitable for resource-constrained environments. Ablation studies demonstrated the roles of key components, including LoRA, multi-axis attention and knowledge distillation, whilst explainable AI techniques ensured the attention to clinically relevant features. This research contributes to the detection of skin cancer with an efficient and accurate deep learning framework for clinical implementation in l