Matrix stochastic game with Q-learning

Authors:

Petro Kravets

Information Systems and Networks Department, Lviv Polytechnic National University, S. Bandery Str., 12, Lviv, 79013, UKRAINE

The model of matrix stochastic game for decision-making in the conditions of uncertainty is developed. The method of Q-learning for stochastic game solving with a priori unknown gains matrices is offered. The formulation of a game problem is executed. The Markovian recurrent method and algorithm for the game solving are described. Results of computer modelling of stochastic game with Q-learning are received and analysed.

stochastic game

uncertainty conditions

Q-learning

Markovian recurrent method

1. Доманcкий В.К. Cтохаcтичеcкие игры / В.К. Доманcкий // Математичеcкие вопроcы кибернетики. — 1988. — № 1. — C. 26 — 49.
2. Fudenberg, D. The Theory of Learning in Games / D. Fudenberg, D.K. Levine. — Cambridge, MA: MIT Press, 1998. — 292 p.
3. Weiss G. Multiagent Systems. A Modern Approach to Distributed Artificial Intelligence / G. Weiss, editor. — Springer Verlag, Berlin, 1996. — 643 p.
4. Wooldridge M. An Introduction to Multiagent Systems / M. Wooldridge. — John Wiley & Sons, 2002. — 366 p.
5. Назин, А.В. Адаптивный выбор вариантов: Рекуррентные алгоритмы / А.В. Назин, А.C. Позняк. — М.: Наука, 1986. — 288 c.
6. Watkins, C.J.C.H. Q-Learning / C.J.C.H. Watkins, P. Dayan // Machine Learning. — Kluwer Academic Publishers, Boston. — 1992. — No. 8. — PP. 279–292.
7. Kaelbling, Leslie. Reinforcement learning: A survey / Leslie Kaelbling, Michael L. Littman, Andrew W. Moore. Journal of Artificial Intelligence Research. — 1996. — No. 4. — PP. 237–285.
8. Sutton, R. S. Reinforcement Learning: An Introduction / Richard S. Sutton, Andrew G. Barto. — MIT Press, 1998. — 322 p.
9. Hu, J. Nash Q-learning for general-sum stochastic games / J. Hu, M. P. Wellman // Machine Learning Research. — 2003. — No. 4. — PP. 1039–1069.
10. Weinberg, M. Best-Response Multiagent Learning in Non-Stationary Environments / Michael Weinberg, Jeffrey S. Rosenschein // AAMAS’04. — New York, USA. — July 19-23, 2004.
11. Подиновcкий В.В. Парето-оптимальные решения многокритериальных задач / В.В. Подиновcкий, В.Д. Ногин. — М.: Наука, 1982. — 256 c.
12. Граничин О.Н. Введение в методы cтохаcтичеcкой аппрокcимации и оценивания: Учеб. поcобие / О.Н. Граничин. — CПб.: Издательcтво C.-Петербургcкого универcитета, 2003. — 131 c.
13. Мулен Э. Теория игр c примерами из математичеcкой экономики / Э. Мулен. — М.: Мир, 1985. — 200 c.