Machine learning methods for control of non-playable characters behaviour in multiplayer rpg

2023;
: pp. 79 - 92
1
Lviv Polytechnic National University, Ukraine
2
Lviv Politechnic National University, Software Department

This article covers the problem of developing a control system for non-player characters in a multiplayer RPG. Commercial projects in the field of videogames and RPG (Role-Playing Game) projects in particular seldom use machine learning models for the implementation of character behaviour. The most common approach is to use primitive preprogrammed rules, or to implement a finite state machine. Such approaches ruin the immersion of playing with real creatures, since various predefined rules make the characters predictable. A good game AI is supposed to give the player an impression of interacting with real characters, that make various decisions, sometimes unpredictable. To achieve this goal, this article covers an approach with using various machine learning models in conjunction with a traditional finite state machine. A videogame developed earlier is used as the basis for problem solution. The article conducts an analysis of the existing works in the field of videogame AI. Next, the implementation of control system is described. This system utilizes a couple selected and successfully trained machine learning models. A multitude of models were tested, eventually a decision tree model and a neural network were selected, since they yielded the best results. The process of development and implementation of a control system involving machine learning models is then described. The approaches of teaching such models are described, and finally the achieved results are analyzed. To gauge the results, different models were compared against eachother in test battles. The decision tree model showed results slightly better than the traditional finite state machine. Meanwhile, the neural network performed significantly better, beating other models far more often. Achieved results can be developed further, utilizing more complex models and improving training methods, which will result in even more sophisticated characters. Presence of such characters in the game will qualitatively improve the gameplay. Obtained system was integrated into the videogame, that may potentially become a commercial product.

  1. Hagelbäck, J., & Johansson, S. J. (2009). A multiagent potential field-based bot for real-time strategy games. International Journal of Computer Games Technology, 2009, 1–10. https://doi.org/10.1155/2009/910819.
  2. Glavin, F. G., & Madden, M. G. (2015). Adaptive shooting for bots in first person shooter games using reinforcement learning. IEEE Transactions on Computational Intelligence and AI in Games, 7(2), 180–192. https://doi.org/10.1109/TCIAIG.2014.2363042.
  3. Karpov, I. V., Schrum, J., & Miikkulainen, R. (2013). Believable bot navigation via playback of human traces. In P. Hingston (Ed.), Believable Bots, 151–170. Springer Berlin Heidelberg. http://dx.doi.org/10.1007/978-3- 642-32323-2_6.
  4. Okun, J. (2016). Evaluating the improvements of starcraft gameplay in the abl agent eisbot by implementing dynamic specificities through an external component. MATEC Web of Conferences, 68, 18001. https://doi.org/10.1051/matecconf/20166818001.
  5. Nicolau, M., Perez-Liebana, D., O’Neill, M., & Brabazon, A. (2017). Evolutionary behavior tree approaches for navigating platform games. IEEE Transactions on Computational Intelligence and AI in Games, 9(3), 227–238. http://dx.doi.org/10.1109/TCIAIG.2016.2543661.
  6. Hagelback, J. (2012). Potential-field based navigation in starcraft. IEEE Conference on Computational In- telligence and Games (CIG), Granada, Spain, 388–393. http://dx.doi.org/10.1109/CIG.2012.6374181.
  7. Perez, D., Nicolau, M., O’Neill, M., & Brabazon, A. (2011). Reactiveness and navigation in computer games: different needs, different approaches. IEEE Conference on Computational Intelligence and Games (CIG'11), Seoul, Korea (South), 273–280. http://dx.doi.org/10.1109/CIG.2011.6032017.
  8. Van Hoorn, N., Togelius, J., Wierstra, D., & Schmidhuber, J. (2009). Robust player imitation using multiobjective evolution. IEEE Congress on Evolutionary Computation, Trondheim, Norway, 652–659. http://dx.doi.org/10.1109/CEC.2009.4983007.
  9. Cerny, V., & Dechterenko, F. (2015). Rogue-like games as a playground for artificial intelligence – evolu- tionary approach. In K. Chorianopoulos, M. Divitini, J. Baalsrud Hauge, L. Jaccheri, R. Malaka (Eds.), Entertainment Computing – ICEC 2015, 261–271. Springer. https://doi.org/10.1007/978-3-319-24589-8_20.
  10. Kempka, M., Wydmuch, M., Runc, G., Toczek, J., & Jaśkowski, W. (2016). ViZDoom: a Doom-based AI research platform for visual reinforcement learning. IEEE Conference on Computational Intelligence and Games (CIG), Santorini, Greece, 1–8. http://dx.doi.org/10.1109/CIG.2016.7860433
  11. OpenJS Foundation (n. d.). Node.js About. https://nodejs.org/en/about/
  12. Photon Storm Ltd. (2021). Phaser 3 API Documentation – Class: Scene. https://photonstorm.github.io/ phaser3-docs/Phaser.Scene.html.