Hechth–Nielsen theorem for a modified neural network with diagonal synaptic connections

: pp. 101-108
Received: September 24, 2018
Revised: December 31, 2018
Accepted: December 31, 2018
Ivan Franko Drogobych State Pedagogical University
Lviv Polytechnic National University
Lviv Polytechnic National University
Ivan Franko Drogobych State Pedagogical University
Lviv Polytechnic National University

The work suggests a modified three-layer neural network with architecture that has only the diagonal synaptic connections between neurons; as a result we obtain the transformed Hecht-Nielsen theorem.  This architecture of a three-layer neural network ($m=2n+1$ is the number of neurons in the hidden layer of the neural network; $n$ is the number of input signals) allows us to approximate the function of $n$ variables, with the given accuracy $\varepsilon>0$, using one aggregation operation, whereas a three-layer neural network that has both diagonal and non-diagonal synaptic connections between neurons approximates the function of $n$ variables by means of two aggregation operations. In addition, the matrix diagonalization of the synaptic    connections leads to a decrease of computing resources and reduces the time of adjustment of the weight coefficients during the training of a neural network.

  1. Kolmogorov A. N.  On the representation of continuous functions of several variables by superpositions of continuous functions of a smaller number of variables. DAN USSR. 108, 2 (1956).
  2. Kolmogorov A. N.  On the representation of continuous functions of several variables by superpositions of continuous functions of one variable and addition.  DAN USSR. 114, 953--956 (1957).
  3. Vitushkin A. G., Khenkin G. M.  Linear superposition of functions.  Russ. Math. Surv. 22, 77--125 (1967).
  4. Slupecki J.  The criterion of completeness of multivalent accounting system.  C. R. Classe III. 32 (1--3), 102--109 (1939).
  5. McCulloch W. S., Pitts W.  A logical calculus of the ideas immanent in nervous activity.  Bulletin of Mathematical Biophysics. 5 (4), 115--133  (1943).
  6. Rosenblatt F.  Principles of Neurodynamics: Perceptrons and the theory of brain mechanisms.  Spartan Books. Washington, DC (1961).
  7. Minsky M., Papert S.  Perceptrons.  Oxford, England: M.I.T. Press. (1969).
  8. Cybenko G.  Approximations by superpositions of sigmoidal functions.  Math. Control Signals Systems. 2, 303--314 (1989).
  9. Funahashi K.  On the approximate realization of continuous mappsngs by neural networks.  Neural Networks. 2 (3), 183--192 (1989).
  10. Hornick K., Stinchcombe M., White H.  Multilayer feedforward networks are universal approximators.  Neural Networks. 2 (5), 359--366 (1989).
  11. HechtNielsen R.  Kolmogorov's mapping neural network existence theorem.  IEEE First Annual Int. Conf. on Neural Networks, San Diego. 3, 11--13 (1987).
  12. Alekseev D. V.  Approximation of functions of several variables by neural networks.  Fundamental and applied mathematics. 15 (3), 9--21 (2009).
  13. Penrose R.  Shadows of the Mind: A Search for the Missing Science of Consciousness.  Oxford University Press, Inc. New York, NY, USA (1994).
  14. Lytvyn V., Vysotska V., Peleshchak I., Rishnyak I., Peleshchak R.  Time dependence of the output signal morphology for nonlinear oscillator neuron based on Van der Pol model.  International Journal of Intelligent Systems and Applications. 4, 8--17 (2018).
  15. Peleshchak R. M., Lytvyn V. V., Peleshchak I. R.  Dynamics of a nonlinear oscillatory neuron under the action of an external non-stationary signal.  Radioelectronics, computer science, management. 4, 97--105 (2017).
  16. Lytvyn V., Peleshchak I., Peleshchak R.  The compression of the input images in neural network that using method diagonalization the matrices of synaptic weight connections.  2017 2nd International Conference on Advanced Information and Communication Technologies (AICT). 66--70 (2017).
  17. Lytvyn V., Peleshchak I., Peleshchak R.  Increase the speed of detection and recognition of computer attacks in combined diagonalized neural networks.  2017 4th International Scientific-Practical Conference "Problems of infocommunications. Science and Technolohy''. 152--155 (2017).
Math. Model. Comput. Vol.6, No.1, pp.101-108 (2019)