In the era of rapid technological advancement, when robotics and intelligent systems are becoming an integral part of everyday life, the importance of developing control systems for mobile robotic platforms using artificial neural networks becomes extremely high and relevant. This field not only has significant practical needs but also holds considerable potential for innovative development. The evolution of modern robotics and computational intelligence has necessitated the creation of more efficient and adaptive mobile robotic systems. A system and tools for controlling mobile robotic platforms using artificial neural networks (ANNs) have been developed in this work. By simulating the workings of a neural system, ANNs enable robots not only to react to input data but also to learn to solve complex tasks and adapt to changes in their environment.
One of the key challenges in mobile platform control is the development of effective and intuitive interfaces that provide convenient and reliable interaction between the user and the robotic system. In this context, the use of hand gestures by humans represents a progressive and promising direction as it allows for the creation of the most natural and efficient means of control. The main task is to create an effective and intuitively understandable system that enables the operator to interact with the robotic platform using natural movements and gestures. As a result, software with a graphical interface for real-time gesture recognition using machine learning has been developed.
The scientific novelty of the approach is the integration of advanced ANNs methods to improve the quality of control and functionality of mobile robotic platforms. The main aspects of scientific novelty include integration with artificial intelligence, interactivity of control, development of robotics mobility, and adaptability to various tasks. The problem addressed in this work lies in the need to develop effective and intuitive control systems for mobile robotic platforms using gesture recognition technologies.
A program based on convolutional neural networks has been developed, which determines the position of the hand and identifies specific gestures such as forward, backward, right and left turns, as well as stopping movement. The implemented technology can be used in various fields of human activity (smart home control, technological solutions for people with physical disabilities, enhancing interactivity in entertainment devices, improving interfaces for interacting with technical equipment).
1. Chen, D., Li, S., & Wu, Q. (2021). A novel supertwisting zeroing neural network with application to mobile robot manipulators. IEEE Transactions on Neural Networks and Learning Systems, 32(4), 1776–1787. https://doi.org/10.1109/TNNLS.2020.2991088
2. Lee, M.-F. R., & Chien, T.-W. (2020). Artificial intelligence and internet of things for robotic disaster response. 2020 International Conference on Advanced Robotics and Intelligent Systems (ARIS), Taipei, Taiwan, 1–6. https://doi.org/10.1109/ARIS50834.2020.9205794
3. Magrin, C. E., & Todt, E. (2019). Multi-sensor fusion method based on artificial neural network for mobile robot self-localization. 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR), and 2019 Workshop on Robotics in Education (WRE), Rio Grande, Brazil, 138–143. https://doi.org/10.1109/LARS-SBR-WRE48964.2019.00032
4. Kwiecień, J. (2022). Selected topics of artificial intelligence in robotics. 2022 7th International Conference on Mechanical Engineering and Robotics Research (ICMERR), Krakow, Poland, 40-44. https://doi.org/10.1109/ICMERR56497.2022.10097788
5. Ostrovka, D., Stasenko, D., & Teslyuk, V. (2022). Autonomous intelligent control system for mobile robotic system. In Proceedings of the IEEE 17th International Conference on Computer Science and Information Technologies, Lviv, Ukraine, 206–209. https://doi.org/10.1109/CSIT56902.2022.10000459
6. Тsmots, I., Tkachenko, R., Teslyuk, V., Opotyak, Y., & Rabyk, V. (2022). Hardware components for nonlinear neuro-like data protection in mobile smart systems. In Proceedings of the IEEE 17th International Conference on Computer Science and Information Technologies, Lviv, Ukraine, 198–202. https://doi.org/10.1109/CSIT56902.2022.10000636
7. Posture and gesture recognition for human-computer interaction. (n.d.). IntechOpen. https://www.intechopen.com/chapters/8712 (Accessed: 22.10.2023).
8. Pavlovic, V. I., Sharma, R., & Huang, T. S. (1997). Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 683-695. https://doi.org/10.1109/34.598226
9. Recognition and classification of sign language. (n.d.). Scielo. Retrieved from: http://www.scielo.org.mx/scielo.php?script=sci_arttext&pid=S1405-55462018000100271 (Accessed: 11.11.2023).
10. Evergreens. (n.d.). Convolutional neural networks: Explained in simple words. Retrieved from: https://evergreens.com.ua/ua/articles/cnn.html
11. Makantasis, K., Karantzalos, K., Doulamis, A., & Doulamis, N. (2015). Deep supervised learning for hyperspectral data classification through convolutional neural networks. In Proceedings of the 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Milan, Italy, 4959–4962. https://doi.org/10.1109/IGARSS.2015.7326945
12. Zinko, R., Teslyuk, V., & Seneta, M. (2023). The evaluation of factors that influence the route formation of the mobile rescue robot. Proceedings of the 7th International Conference on Computational Linguistics and Intelligent Systems. Kharkiv, Ukraine. In CEUR Workshop Proceedings, 3403, 570‑581.
13. Borkivskyi, B. P., & Teslyuk, V. M. (2023). Application of neural network tools for object recognition in mobile systems with obstacle avoidance. Scientific Bulletin of UNFU, 33(4), 84-89. https://doi.org/10.36930/40330412
14. Teslyuk, V., Borkivskyi, B., & Alshawabkeh, H. A. (2022). Models and means of object recognition using artificial neural networks. Proceedings of the 4th International Workshop MoMLeT&DS 2022. Leiden-Lviv, the Netherlands-Ukraine. In CEUR Workshop Proceedings, Vol. 3312, 241‑251.
15. Zinko, R. V., Korendiy, V. M., Tesliuk, V. M., Demchuk, I. B., Kazymyra, I. Ya., & Ostrovka, D. V. (2022). Frequency analysis of the shaft of the electromechanical drive of the small mobile robot. Industrial Process Automation in Engineering and Instrumentation, 56, 27‑38. https://doi.org/10.23939/istcipa2022.56.027
16. Zinko, R. V., Teslyuk, V. M., Kazymyra, I. Ya., & Ostrovka, D. V. (2022). A model for improving the strength characteristics of the electromechanical drive of a mobile robot. Ukrainian Journal of Information Technology, 4(2), 80–85. https://doi.org/10.23939/ujit2022.02.080