Distributed Adaptation of the Functional Integration Structure of a Multi-agent System in a Dual-tasking Environment

2019;
: pp. 65 - 74
1
Lviv Polytechnic Natioinal University

The paper considers the problem of distributed adaptation of the functional integration structure of a multi- agent system in a dual-tasking environment from the point of view of organizing multi-agent search and use of the functional emergence effect provided by different structures of functional integration. The considered problem belongs to a wider class of problems of structural adaptation and self-organization. Models of functional integration, in particular, models based on general quantitative characteristics of the functional roles distribution of agents and models based on local qualitative characteristics of the functional roles distribution of agents, taking into account the specifics of functional links established between agents have been considered in the paper. The problems of the distributed adaptation of the functional integration structure have been analyzed, including the problem of the functional specialization of agents in a multitasking environment. Various ways of organizing structural changes have been considered, including multi- agent parametric adaptation based on a local structural parameter. Multi-agent structural adaptation based on reinforcement learning methods, in particular, multi-agent structural adaptation based on the normalized exponential function method (MSA-softmax) and multi-agent structural adaptation based on the upper confidence bound method (MSA-UCB) has been proposed. The distributed adaptation methods simulation results have been presented, which showed the advantage of multi-agent structural adaptation over multi- agent parametric adaptation.

[1] Multiagent Systems, by Gerhard  Weiss (Editor), 2nd edition, The MIT Press, 2013. - 920 p.

[2] Michael Wooldridge, An Introduction to MultiAgent Systems, 2nd edition, Wiley, 2009. - 484 p.

[3] Yoav Shoham, Kevin Leyton-Brown, Multiagent Systems: Algorithmic,Game-Theoretic, and Logical Foundations, Cambridge University Press, 2008. - 504 p.

[4] Multi-Agent Systems: Simulation and Applications, by Adelinde M. Uhrmacher (Editor), Danny Weyns (Editor), CRC Press, 2009. - 566 p.

[5 Caruana, R. (1997). Multitask Learning. Machine Learning, 28(1), 41–75

[6] Thung, K., Wee, C. A brief review on multi-task learning, Multimedia Tools and Applications, v.77, 2018. - pp. 29705–29725

[7]  Zhang, Y., & Yeung, D.-Y. (2012, March 15). A Convex Formulation for Learning Task Relationships in Multi- Task Learning, in Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI 2010), Catalina Island,CA, USA, July 8-11, 2010. - pp. 733-742

[8] Ando, R.; Zhang, T. (2005) A framework for learning predictive structures from multiple tasks and unlabeled data. The Journal of Machine Learning Research. 6: 1817–1853

[9] Argyriou, A.; Evgeniou, T.; Pontil, M. (2008) Convex multi-task feature learning. Machine Learning. 73 (3): 243–272

[10] Swersky, K., Snoek, J., & Adams, R. P. (2013) Multi-task bayesian optimization. Advances in neural information processing systems. - pp. 2004-2012

[11] Ong, Y. S., & Gupta, A. (2016). Evolutionary multitasking: a computer science view of cognitive multitasking. Cognitive Computation, 8(2). - pp. 125-142

[12] Cheng, M. Y., Gupta, A., Ong, Y. S., & Ni, Z. W. (2017). Coevolutionary multitasking for concurrent global optimization: With case studies in complex engineering design. Engineering Applications of Artificial Intelligence, v.64. - pp. 13- 24

[13] Bao, L., Qi, Y., Shen, M., Bu, X., Yu, J., Li, Q., & Chen, P. (2018, June). An Evolutionary Multitasking Algorithm for Cloud Computing Service Composition. In World Congress on Services. Springer, Cham. - pp. 130-144

[14] Uchibe, E., Kato, T., Hosoda, K., & Asada, M. (2001). Dynamic task  assignment  in  a  multiagent/multitask environment based on module conflict resolution. Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation, 4, vol.4., 3987-3992 

[15]  Alessandro  Agnetis,   Jean-Charles   Billaut, Stanisław Gawiejnowicz,  Dario Pacciarelli, Ameur Soukhal, Multiagent scheduling: Models and algorithms, Springer; 2014. - 271 p.

[16] Li, Shisheng & Chen, Ren-Xia & Tian, Ji. (2019). Multitasking scheduling problems with two competitive agents. Engineering Optimization. 1-17

[17] Botchkaryov A., Golembo V., Applying intelligent technologies of data collection to autonomous cyber-physical systems, Transactions on Computer systems and networks, Lviv Polytechnic National University Press, No. 830, 2015. – pp.7-11 (in Ukrainian)

[18]   Melnyk  A.,  Golembo  V.,  Botchkaryov  A.,  The new principles of designing configurable smart sensor networks based on intelligent agents, Transactions on Computer systems and networks, Lviv Polytechnic National University Press, No. 492, 2003. – pp.100-107 (in Ukrainian)

[19] Botchkaryov A., Collective behavior of mobile intelligent agents solving the autonomous distributed exploration task, Transactions on Computer systems and networks, Lviv Polytechnic National University Press, No. 546, 2005. – pp.12-17 (in Ukrainian)

[20] Botchkaryov A., Structural adaptation of the autonomous distributed sensing and computing systems, Transactions on Computer systems and  networks,  Lviv Polytechnic National University Press, No. 688, 2010. – pp.16-22 (in Ukrainian)

[21] Botchkaryov  A.,  The  problem  of  organizing adaptive sensing and computing processes in autonomous distributed systems, Transactions on Computer systems and networks, Lviv Polytechnic National University Press, No. 745, 2012. – pp.20-26 (in Ukrainian)

[22] Serge Kernbach, Structural Self-Organization in Multi-Agents and Multi-Robotic Systems, Logos Verlag, 2008. - 250 p.

[23] Preisler, Thomas & Renz, Wolfgang. (2015). Structural Adaptations for Self-Organizing Multi-Agent Systems, The Seventh International Conference on Adaptive and Self- Adaptive Systems and Applications (ADAPTIVE 2015), At Nice, France

[24] Jiao, W., & Sun, Y. (2016). Self-adaptation of multi-agent systems in dynamic environments based on experience exchanges. Journal of Systems and Software, 122, 165–179 

[25] Richard  S.  Sutton,  Andrew  G.  Barto, Reinforcement Learning: An Introduction, 2nd edition, The MIT Press, 2018. - 552 p.

[26] L.P. Kaelbling, Michael  L. Littman, and Andrew W. Moore, Reinforcement learning: A survey. Journal of AI Research, N4, 1996. - pp.237-285

[27]     Csaba Szepesvari, Algorithms for Reinforcement Learning, Morgan and Claypool Publishers, 2010. - 104 p.

[28] Maxim Lapan,  Deep Reinforcement   Learning   Hands-On,   Packt Publishing, 2018. - 546 p.

[29] H. M. Schwartz, Multi-Agent Machine Learning: A Reinforcement Approach, Wiley, 2014. - 256 p.

[30] Auer, P. Using upper confidence bounds for online learning. Proceedings 41st Annual Symposium on Foundations of Computer Science. IEEE Comput. Soc., 2000. - pp. 270–279

[31] Auer, P., Cesa-Bianchi, N., Fischer, P., Finite-time analysis of the multiarmed bandit problem. Machine learning, 47(2-3), 2002. - pp.235–256

[32] Tsetlin, M. L., Automaton Theory and Modeling of Biological Systems. Academic Press, New York, 1973. – 288 p.

[33]  Varshavsky,  V.  I.,  Collective   behavior   of automata, Moscow, Nauka, 1973. - 408 p.

[34] Narendra, K. and Thathachar, M. A. L., Learning Automata: An Introduction, 2nd ed., Dover Publications, 2013. - 496 p.

 Alexey Botchkaryov was born in 1975 in Lviv, Ukraine. He received the B.S. and the M.S. degrees in Computer Engineering at Lviv Polytechnic National University in 1998 and the Ph.D. degree in Computer Systems and Components at Lviv Polytechnic National University in 2019. He has been doing scientific and research  work since 1994. He is the author of a book (with coauthors) and more than 30 articles. Currently, he is a Senior Lecturer at the Computer Engineering Department and a Research Fellow at the Intelligent Systems Laboratory, Lviv Polytechnic National University. His research interests include self-organization in complex systems, structural adaptation, intelligent information-gathering agents and multi-agent systems.