1. SISN
  2. All Volumes and Issues
  3. Volume 14, 2023
  4. Identification of Birds' Voices Using Convolutional Neural Networks Based on Stft and Mel Spectrogram

Identification of Birds' Voices Using Convolutional Neural Networks Based on Stft and Mel Spectrogram

SISN.
2023;
Volume 14
: pp. 297 - 311
https://doi.org/10.23939/sisn2023.14.297
Authors:
  1. Oksana Honsor
  2. Yuriy Gonsor
1
Lviv Polytechnic National University, Specialized Computer Systems Department
2
Lviv Polytechnic National University, Artificial Intelligence Systems Department

Threats to the climate and global changes in ecological processes remain an urgent problem throughout the world. Therefore, it is important to constantly monitor these changes, in particular, using non-standard approaches. This task can be implemented on the basis of research on bird migration information. One of the effective methods of studying bird migration is the auditory method, which needs improvement. That is why building a model based on machine learning methods that will help to accurately identify the presence of bird voices in an audio file for the purpose of studying bird migrations from a given area is an urgent problem. This paper examines ways of building a machine learning model based on the analysis of spectrograms, which will help to accurately identify the presence of bird voices in an audio file for the purpose of studying the migration of birds in a certain area. The research involves the collection and analysis of audio files that can be used to identify characteristics that will identify the sound of the files as birdsong or the absence of sound in the file. The use of the CNN model for the classification of the presence of bird voices in an audio file is demonstrated. Special attention is paid to the effectiveness and accuracy of the CNN model in the classification of sounds in audio files, which allows you to compare and choose the best classifier for a given type of file and model. Analysis of the effectiveness and accuracy of the CNN model in the classification of sounds in audio files showed that the use of Mel-spectrograms is better than the use of STFT-spectrograms for studying the classification of the presence of bird sounds in the environment. The classification accuracy of the model trained on the basis of Mel spectrograms was 72 %, which is 8 % better than the accuracy of the model trained on STFT spectrograms.

machine learning
sound identification
spectrogram
convolutional neural network
  1. Ghosh A., Sufian A., Sultana F., Chakrabarti A. & Debashis De. (2020). Fundamental Concepts of Convolutional Neural Network. Recent Trends and Advances in Artificial Intelligence and Internet of Things, 519–567. DOI:10.1007/978-3-030-32644-9_36.
  2. Krizhevsky A., Sutskever I., & Hinton G. E. (2012). Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, 1097–1105.
  3. Sultana F., Sufian A., & Dutta P. (2019). A review of object detection models based on convolutional neural network. CoRR, abs/1905.01614. DOI:10.1007/978-981-15-4288-6_1.
  4. Sultana F., Sufian A., & Dutta P. (2018). Advancements in image classification using convolutional neural network. In 2018 Fourth International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), 122–129.
  5. Everingham M., Van Gool L., Williams C. K. I., Winn J. & Zisserman A. (2010). The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 88(2), 303–338. DOI:10.1007/s11263-009-0275- 4.
  6. Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., & Rabinovich A. (2015). Going deeper with convolutions. In The IEEE Conference on ComputerVision and Pattern Recognition (CVPR). DOI: 10.48550/arXiv.1409.4842.
  7. Shelhamer E., Long J., & Darrell T. (2015). Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 39(4), 640–651. DOI: 10.1109/CVPR.2015.7298965.
  8. Dennis J. W. (2014). Sound event recognition in unstructured environments using spectrogram image processing.                     Doctoral            thesis, Nanyang Technological University,                                                                    Singapore. DOI: 10.32657/10356/59272
  9. Mesaros A., Heittola T., Eronen A., & Virtanen T. (2010). Acoustic event detection in real life recordings.Proceedings of the European Signal Processing Conference (EUSIPCO), 1267–1271.
  10. Tsau E., Chachada S., & Kuo C.-C. J. (2012). Content/Context-Adaptive Feature Selection for Environmental Sound Recognition. Proceedings of the Asia Pacific Signal & Information Processing Association (APSIPA).
  11. Zhang Z. and Schuller B. Semi-supervised learning helps in sound event classification. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), IEEE, 333–336. March, 2012.
  12. Maccagno A., Mastropietro A., Mazziotta U., Scarpiniti M., Lee Y.-Ch. & Uncini A. (2021). A CNN Approach for Audio Classification in Construction Sites. Progresses in Artificial Intelligence and Neural Systems, 371– 381. DOI: 10.1007/978-981-15-5093-5_33.
  13. Ekpezu A., Wiafe I., Katsriku F. & Yaokumah W. (2021). Using deep learning for acoustic event classification: The case of natural disasters. The Journal of the Acoustical Society of America, 149(4): 292. DOI: 10.1121/10.0004771.
  14. Khamparia A., Gupta D., Nguyen N. G., Khanna A., Pandey B., & Tiwari P. (2019). Sound classification using convolutional neural network and tensor deep stacking network, IEEE Access, 7(1), 7717–7727. DOI: 10.1109/ACCESS.2018.2888882.
  15. Zhang, T., Lee, Y.-C., Scarpiniti, M., Uncini, A. (2018). A supervised machine learning-based sound identification for construction activity monitoring and performance evaluation. Proceedings of 2018 Construction Research Congress (CRC 2018), New Orleans, Louisiana, USA, 358–366.
  16. Kons Z., Toledo-Ronen O. (2013). Audio Event Classification Using Deep Neural Networks. Proc. Interspeech 2013, 1482–1486. DOI: 10.21437/Interspeech.2013-384.
  17. Lee H., Grosse R., Ranganath R., & Ng A.Y. (2011). Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks. Communications of the ACM, Vol. 54, No. 10, 95–103. DOI: 10.1145/2001269.2001295.
  18. Gartzman D. Getting to Know the Mel Spectrogram. Towards Data Science. August, 2019. Retrieved from: https://towardsdatascience.com/getting-to-know-the-mel-spectrogram-31bca3e2d9d0 (date of access: 20.09.2023)
  19. Papia Nandi – CNNs for audio classification. A primer in deep learning for audio classification using TensorFlow. Towards Data Science. Murch, 2021. Retrieved from: https://towardsdatascience.com/cnns-for- audio- classification-6244954665ab (date of access: 16.09.2023)
  20. Chollet, F. Deep Learning with Python (2018), v. 361, New York: Manning.
  21. SHANTAMVIJAYPUTRA - Bird Voice Detection Dataset. Retrieved from: https://www.kaggle.com/datasets/vshantam/bird-voice-detection (date of access: 15.05.2023)