: 8-14
Received: June 12, 2021
Accepted: June 01, 2021

Цитування за ДСТУ: Дем'янець Т. В., Федасюк Д. В. Застосування згорткової нейронної мережі для виявлення меланоми за зо­бра­женням новоутворення на мобільному пристрої. Український журнал інформаційних технологій. 2021, т. 3, № 1. С. 08–14.

Citation APA: Demianets, T. V., & Fedasyuk, D. V. (2021). Application of convolutional neural network for detection of melanoma using skin lesion image on mobile device. Ukrainian Journal of Information Technology, 3(1), 08–14.

Lviv Polytechnic National University, Lviv, Ukraine
Lviv Polytechnic National University, Lviv, Ukraine

A melanoma is the deadliest skin cancer, so early diagnosis can provide a positive prognosis for treatment. Modern methods for early detecting melanoma on the image of the tumor are considered, and their advantages and disadvantages are analyzed. The article demonstrates a prototype of a mobile application for the detection of melanoma on the image of a mole based on a convolutional neural network, which is developed for the Android operating system. The mobile application contains melanoma detection functions, history of the previous examinations and a gallery with images of the previous examinations grouped by the location of the lesion. The HAM10000-based training dataset has been supplemented with the images of melanoma from the archive of The International Skin Imaging Collaboration to eliminate class imbalances and improve network accuracy. The search for existing neural networks that provide high accuracy was conducted, and VGG16, MobileNet, and NASNetMobile neural networks have been selected for research. Transfer learning and fine-tuning has been applied to the given neural networks to adapt the networks for the task of skin lesion classification. It is established that the use of these techniques allows to obtain high accuracy of the neural network for this task. The process of converting a convolutional neural network to an optimized Flatbuffer format using TensorFlow Lite for placement and use on a mobile device is described. The performance characteristics of the selected neural networks on the mobile device are evaluated according to the classification time on the CPU and GPU and the amount of memory occupied by the file of a single network is compared. The neural network file size was compared before and after conversion. It has been shown that the use of the TensorFlow Lite converter significantly reduces the file size of the neural network without affecting its accuracy by using an optimized format. The results of the study indicate a high speed of application and compactness of networks on the device, and the use of graphical acceleration can significantly decrease the image classification time of the tumor. According to the analyzed parameters, NASNetMobile was selected as the optimal neural network to be used in the mobile application of melanoma detection.

  1. Abbasi, N. R., Shaw, H. M., Rigel, D. S., Friedman, R. J., McCarthy, W. H., Osman, I., Kopf, A. W., & Polsky, D. (2004). Early Diagnosis of Cutaneous Melanoma. JAMA, 292(22), 2771–2776.
  2. Adding metadata to TensorFlow Lite models. (2021). TensorFlow.
  3. Bakheet, S. (2017). An SVM Framework for Malignant Melanoma Detection Based on Optimized HOG Features. Computation, 5(4), 4.
  4. Codella, N., Cai, J., Abedini, M., Garnavi, R., Halpern, A., Smith, J.R. (2015). Deep Learning, Sparse Coding, and SVM for Melanoma Recognition in Dermoscopy Images. In: Zhou L., Wang L., Wang Q., Shi Y. (Eds) Machine Learning in Medical Imaging. MLMI 2015. Lecture Notes in Computer Science, vol. 9352. Springer, Cham.
  5. Deng, J., Dong, W., Socher, R., Li, L. J., Kai Li, & Li Fei-Fei. (2009). ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248–255.
  6. Hussain, M., Bird, J. J., Faria, D. R. (2019). A Study on CNN Transfer Learning for Image Classification. In: Lotfi A., Bouchachia H., Gegov A., Langensiepen C., McGinnity M. (Eds) Advances in Computational Intelligence Systems. UKCI 2018. Advances in Intelligent Systems and Computing, vol. 840. Springer, Cham.
  7. Ignatov, A., et al. (2019). AI Benchmark: Running Deep Neural Networks on Android Smartphones. In: Leal-Taixé L., Roth S. (Eds.) Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science, vol. 11133. Springer, Cham.
  8. Kasmi, R., & Mokrani, K. (2016). Classification of malignant melanoma and benign skin lesions: implementation of automatic ABCD rule. IET Image Processing, 10(6), 448–455.
  9. Miller, A. J., & Mihm, M. C. (2006). Melanoma. New England Journal of Medicine, 355(1), 51–65.
  10. Mustafa, S., Dauda, A. B., & Dauda, M. (2017). Image processing and SVM classification for melanoma detection. 2017 International Conference on Computing Networking and Informatics (ICCNI), 1–5.
  11. Nasr-Esfahani, E., Samavi, S., Karimi, N., Soroushmehr, S., Jafari, M., Ward, K., & Najarian, K. (2016). Melanoma detection by analysis of clinical images using convolutional neural network. 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).
  12. Ottom, M. A. (2019). Convolutional neural network for diagnosing skin cancer. International Journal of Advanced Computer Science and Applications, 10(7).
  13. Powers, D. M. W. (2011). Evaluation: From precision, recall and F-measure to ROC, informedness, markedness & correlation. Journal of Machine Learning Technologies, 2(1), 37–63.
  14. Raut, N., Shah, A., Vira, S., & Sampat, H. (2018). A study on different techniques for skin cancer detection. International Research Journal of Engineering and Technology (IRJET), 5(3), 613–617.
  15. Refianti, R., Benny, A., & Poetri, R. (2019). Classification of Melanoma Skin Cancer using Convolutional Neural Network. International Journal of Advanced Computer Science and Applications, 10(3), 409–417.
  16. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. (2018). MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4510–4520.
  17. Simonyan, K., Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
  18. Sultana F., Sufian A., & Dutta P. (2018). Advancements in Image Classification using Convolutional Neural Network. 2018 Fourth International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN).
  19. TensorFlow Lite guide. (2021). TensorFlow.
  20. Tschandl, P., Rosendahl, C., & Kittler, H. (2018). The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5(1).
  21. Yu, C., Yang, S., Kim, W., Jung, J., Chung, K.-Y., Lee, S. W., & Oh, B. (2018). Acral melanoma detection using a convolutional neural network for dermoscopy images. PLOS ONE, 13(3).
  22. Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018). Learning Transferable Architectures for Scalable Image Recognition. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8697–8710.