1. UJIT
  2. All Volumes and Issues
  3. Volume 3 · Number 1
  4. APPLICATION OF CONVOLUTIONAL NEURAL NETWORK FOR DETECTION OF MELANOMA USING SKIN LESION IMAGE ON MOBILE DEVICE

APPLICATION OF CONVOLUTIONAL NEURAL NETWORK FOR DETECTION OF MELANOMA USING SKIN LESION IMAGE ON MOBILE DEVICE

UJIT.
2021;
Volume 3, Number 1
: 8-14
https://doi.org/10.23939/ujit2021.03.008
Received: June 12, 2021
Accepted: June 01, 2021

Цитування за ДСТУ: Дем'янець Т. В., Федасюк Д. В. Застосування згорткової нейронної мережі для виявлення меланоми за зо­бра­женням новоутворення на мобільному пристрої. Український журнал інформаційних технологій. 2021, т. 3, № 1. С. 08–14.

Citation APA: Demianets, T. V., & Fedasyuk, D. V. (2021). Application of convolutional neural network for detection of melanoma using skin lesion image on mobile device. Ukrainian Journal of Information Technology, 3(1), 08–14. https://doi.org/10.23939/ujit2021.03.008

Authors:
  1. T. V. Demianets
  2. D. V. Fedasyuk
1
Lviv Polytechnic National University, Lviv, Ukraine
2
Lviv Polytechnic National University, Lviv, Ukraine

A melanoma is the deadliest skin cancer, so early diagnosis can provide a positive prognosis for treatment. Modern methods for early detecting melanoma on the image of the tumor are considered, and their advantages and disadvantages are analyzed. The article demonstrates a prototype of a mobile application for the detection of melanoma on the image of a mole based on a convolutional neural network, which is developed for the Android operating system. The mobile application contains melanoma detection functions, history of the previous examinations and a gallery with images of the previous examinations grouped by the location of the lesion. The HAM10000-based training dataset has been supplemented with the images of melanoma from the archive of The International Skin Imaging Collaboration to eliminate class imbalances and improve network accuracy. The search for existing neural networks that provide high accuracy was conducted, and VGG16, MobileNet, and NASNetMobile neural networks have been selected for research. Transfer learning and fine-tuning has been applied to the given neural networks to adapt the networks for the task of skin lesion classification. It is established that the use of these techniques allows to obtain high accuracy of the neural network for this task. The process of converting a convolutional neural network to an optimized Flatbuffer format using TensorFlow Lite for placement and use on a mobile device is described. The performance characteristics of the selected neural networks on the mobile device are evaluated according to the classification time on the CPU and GPU and the amount of memory occupied by the file of a single network is compared. The neural network file size was compared before and after conversion. It has been shown that the use of the TensorFlow Lite converter significantly reduces the file size of the neural network without affecting its accuracy by using an optimized format. The results of the study indicate a high speed of application and compactness of networks on the device, and the use of graphical acceleration can significantly decrease the image classification time of the tumor. According to the analyzed parameters, NASNetMobile was selected as the optimal neural network to be used in the mobile application of melanoma detection.

Android application
tumor images
artificial intelligence
machine learning
transfer learning
fine tuning
  1. Abbasi, N. R., Shaw, H. M., Rigel, D. S., Friedman, R. J., McCarthy, W. H., Osman, I., Kopf, A. W., & Polsky, D. (2004). Early Diagnosis of Cutaneous Melanoma. JAMA, 292(22), 2771–2776. https://doi.org/10.1001/jama.292.22.2771
  2. Adding metadata to TensorFlow Lite models. (2021). TensorFlow. https://www.tensorflow.org/lite/guide
  3. Bakheet, S. (2017). An SVM Framework for Malignant Melanoma Detection Based on Optimized HOG Features. Computation, 5(4), 4. https://doi.org/10.3390/computation5010004
  4. Codella, N., Cai, J., Abedini, M., Garnavi, R., Halpern, A., Smith, J.R. (2015). Deep Learning, Sparse Coding, and SVM for Melanoma Recognition in Dermoscopy Images. In: Zhou L., Wang L., Wang Q., Shi Y. (Eds) Machine Learning in Medical Imaging. MLMI 2015. Lecture Notes in Computer Science, vol. 9352. Springer, Cham. https://doi.org/10.1007/978-3-319-24888-2_15
  5. Deng, J., Dong, W., Socher, R., Li, L. J., Kai Li, & Li Fei-Fei. (2009). ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248–255. https://doi.org/10.1109/cvpr.2009.5206848
  6. Hussain, M., Bird, J. J., Faria, D. R. (2019). A Study on CNN Transfer Learning for Image Classification. In: Lotfi A., Bouchachia H., Gegov A., Langensiepen C., McGinnity M. (Eds) Advances in Computational Intelligence Systems. UKCI 2018. Advances in Intelligent Systems and Computing, vol. 840. Springer, Cham. https://doi.org/10.1007/978-3-319-97982-3_16
  7. Ignatov, A., et al. (2019). AI Benchmark: Running Deep Neural Networks on Android Smartphones. In: Leal-Taixé L., Roth S. (Eds.) Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science, vol. 11133. Springer, Cham. https://doi.org/10.1007/978-3-030-11021-5_19
  8. Kasmi, R., & Mokrani, K. (2016). Classification of malignant melanoma and benign skin lesions: implementation of automatic ABCD rule. IET Image Processing, 10(6), 448–455. https://doi.org/10.1049/iet-ipr.2015.0385
  9. Miller, A. J., & Mihm, M. C. (2006). Melanoma. New England Journal of Medicine, 355(1), 51–65. https://doi.org/10.1056/nejmra052166
  10. Mustafa, S., Dauda, A. B., & Dauda, M. (2017). Image processing and SVM classification for melanoma detection. 2017 International Conference on Computing Networking and Informatics (ICCNI), 1–5. https://doi.org/10.1109/iccni.2017.8123777
  11. Nasr-Esfahani, E., Samavi, S., Karimi, N., Soroushmehr, S., Jafari, M., Ward, K., & Najarian, K. (2016). Melanoma detection by analysis of clinical images using convolutional neural network. 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). https://doi.org/10.1109/embc.2016.7590963
  12. Ottom, M. A. (2019). Convolutional neural network for diagnosing skin cancer. International Journal of Advanced Computer Science and Applications, 10(7). https://doi.org/10.14569/ijacsa.2019.0100746
  13. Powers, D. M. W. (2011). Evaluation: From precision, recall and F-measure to ROC, informedness, markedness & correlation. Journal of Machine Learning Technologies, 2(1), 37–63. https://doi.org/10.9735/2229-3981
  14. Raut, N., Shah, A., Vira, S., & Sampat, H. (2018). A study on different techniques for skin cancer detection. International Research Journal of Engineering and Technology (IRJET), 5(3), 613–617.
  15. Refianti, R., Benny, A., & Poetri, R. (2019). Classification of Melanoma Skin Cancer using Convolutional Neural Network. International Journal of Advanced Computer Science and Applications, 10(3), 409–417. https://doi.org/10.14569/ijacsa.2019.0100353
  16. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. (2018). MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4510–4520. https://doi.org/10.1109/CVPR.2018.00474
  17. Simonyan, K., Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv. https://arxiv.org/abs/1409.1556
  18. Sultana F., Sufian A., & Dutta P. (2018). Advancements in Image Classification using Convolutional Neural Network. 2018 Fourth International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN). https://doi.org/10.1109/icrcicn.2018.8718718
  19. TensorFlow Lite guide. (2021). TensorFlow. https://www.tensorflow.org/lite/guide
  20. Tschandl, P., Rosendahl, C., & Kittler, H. (2018). The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5(1). https://doi.org/10.1038/sdata.2018.161
  21. Yu, C., Yang, S., Kim, W., Jung, J., Chung, K.-Y., Lee, S. W., & Oh, B. (2018). Acral melanoma detection using a convolutional neural network for dermoscopy images. PLOS ONE, 13(3). https://doi.org/10.1371/journal.pone.0193321
  22. Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018). Learning Transferable Architectures for Scalable Image Recognition. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8697–8710. https://doi.org/10.1109/CVPR.2018.00907