reinforcement learning

The problem of organizing data collection processes in autonomous distributed systems has been considered, in particular, in autonomous mobile cyber-physical systems and autonomous distributed environmental monitoring systems. A model of decentralized control of adaptive data collection processes based on the principle of equilibrium has been proposed. Using this model, the problem of coordinating joint collective actions of adaptive data collection processes is studied from the point of view of finding an effective scheme for their complementarity in the absence of a control center.

As a result of the analytical review, it was established that smart sensor units are one of the main components of the cyber—physical system. One of the tasks, which have been entrusted to such units, are targeting and tracking of movable objects. The algorithm of targeting on such objects using observation equipment has been considered. This algorithm is able to continuously monitor observation results, predict the direction with the highest probability of movement and form a set of commands to maximize the approximation of a moving object to the center of an information frame.

A method of structural adaptation of data collection processes has been developed based on reinforcement learning of the decision block on the choice of actions at the structural and functional level subordinated to it, which provides a more efficient distribution of measuring and computing resources, higher reliability and survivability of information collection subsystems of an autonomous distributed system compared to methods of parametric adaptation.

In this article, a fuzzy controller tuned by reinforcement learning is proposed.  The developed algorithm utilizes a fuzzy logic theory and a reinforcement learning for fine-tuning parameters of the membership function for the fuzzy controller.  Apart from the fuzzy controller developed, a fuzzy corrector of reference input (set-point) signal to the controller is applied.  The fuzzy corrector changes the input (reference) signal of the system and takes into account an original reference input and type of external disturbances.

The model of decentralized control of adaptive data collection processes has been developed based on the equilibrium concept, which is used to study the problem of coordinating joint collective actions from the point of view of finding an effective scheme for complementing the individual actions of data collection processes in the absence of a control center.