1. CDS
  2. All Volumes and Issues
  3. Volume 1, Number 1, 2019
  4. Hardware Implementation of Parallelized Fuzzy Adaptive Resonance Theory Neural Network

Hardware Implementation of Parallelized Fuzzy Adaptive Resonance Theory Neural Network

CDS.
2019;
Volume 1, Number 1
: pp. 1 - 11
https://doi.org/10.23939/cds2019.01.001
Authors:
  1. Pavlo Tymoshchuk
  2. S. Shatny
1
Lviv Polytechnic National University, Department of Computer Aided Design Systems
2
Lviv Polytechnic National University, Ukraine

A hardware implementation design of parallelized fuzzy Adaptive Resonance Theory neural network is described and simulated. Parallel category choice and resonance are implemented in the network. Continuous-time and discrete-time winner-take-all neural circuits identifying the largest of M inputs are used as the winner-take-all units. The continuous-time circuit is described by a state equation with a discontinuous right-hand side. The discrete-time counterpart is governed by a difference equation. Corresponding functional block-diagrams of the circuits include M feed-forward hard- limiting neurons and one feedback neuron, which is used to compute the dynamic shift of inputs. The circuits combine arbitrary finite resolution of inputs, high convergence speed to the winner-take-all operation, low computational and hardware implementation complexity, and independence of initial conditions. The circuits are also used for finding elements of input vector with minimal/maximal values to normalize them in the range [0,1].

functional block-diagram
fuzzy Adaptive Resonance Theory
neural network
category choice
winner-take-all
state equation with a discontinuous right-hand side
  1. Carpenter G. A., Grossberg S., and Rosen D. B., “Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System”, Neural Networks, vol. 4, no. 6, pp. 759–771, Jun. 1991. https://doi.org/10.1016/0893-6080(91)90056-B
  2. Grossberg S. and Levine D. S., “Neural dynamics of attentionally modulated Pavlovian conditioning: blocking, inter-stimulus interval, and secondary reinforcement”, Applied Optics, vol. 26, no.23, pp. 5015–5030, Dec. 1987. https://doi.org/10.1364/AO.26.005015
  3. Wunsch II D. C., “ART properties of interest in engineering applications”, in Proc. Int. Joint Conf. Neural Networks, 2009, pp. 3380–3383.
  4. Grossberg S., “Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world”, Neural Networks, vol. 37, pp. 1–47, Jan. 2013. https://doi.org/10.1016/j.neunet.2012.09.017
  5. Tan A.-H., Carpenter G. A., and Grossberg S., “Intelligence through interaction: Towards a unified theory for learning”, in Proc. 4th Int. Symp. Neural Networks, LNCS 4491, 2007, pp. 1094–1103. https://doi.org/10.1007/978-3-540-72383-7_128
  6. Martínez-Zarzuela M., Pernas F., Díez Higuera J., and Antón-Rodríguez M., “Fuzzy ART neural network parallel computing on the GPU”, in Proc. 9th Int. Work-Conf. Art. Neural Networks, LNCS 4507, 2007, pp. 463–470. https://doi.org/10.1007/978-3-540-73007-1_57
  7. Martnez-Zarzuela M., Pernas F., Pablos A. de, Rodrguez M., Higuera J., Giralda D., and Ortega D., “Adaptive Resonance Theory fuzzy networks parallel computation using CUDA”, in Proc. 10th Int. Work-Conf. Art. Neural Networks, LNCS 5517, 2009, pp. 149–156. https://doi.org/10.1007/978-3-642-02478-8_19
  8. Ho C. S., Liou J. J., Georgiopoulos M., Heileman G. L., and Christodoulou C., “Analogue circuit design and implementation of an adaptive resonance theory (ART) neural network architecture”, Int. J. Electronics, vol. 76, no 2, pp. 271–291, Apr. 1994. https://doi.org/10.1080/00207219408925926
  9. Tsay S. W. and Newcomb R. W., “VLSI implementation of ART1 memories”, IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 214–221, March 1991. https://doi.org/10.1109/72.80330
  10. Wunsch D. C., Caude U. T. P., Capps C. D., Marks R. J., and Falk R. A., “An optoelectronic implementation of the adaptive resonance fuzzy neural network”, IEEE Trans. Neural Networks, vol. 4, no. 4, pp. 673–684, July 1993. https://doi.org/10.1109/72.238321
  11. Wunsch II D. C., Morris D. J., McGann R. K., and Caudell T. P., “Photorefractive Adaptive Resonance Neural Network”, Applied Optics, vol. 32, no. 8, pp. 1399–1407, Mar. 1993. https://doi.org/10.1364/AO.32.001399
  12. Serrano-Gotarredona T. and Linares-Bamanco B., “A modified ARTI algorithm more suitable for VLSI implementations”, Neural Networks, vol. 9, no. 6, pp. 1025–1043, Aug. 1996. https://doi.org/10.1016/0893-6080(95)00145-X
  13. Serrano-Gotarredona T. and Linares-Bamanco B., “A real-time clustering microchip neural engine”, IEEE Trans. VLSI Systems, vol. 4 no. 2, pp. 195–209, June 1996. https://doi.org/10.1109/92.502192
  14. Carpenter G. A. and Grossberg S., “A massively parallel architecture for a self-organizing neural pattern recognition machine”, Computer Vision, Graphics, and Image Processing, vol. 37, no. 1, pp. 54–115, Jan. 1987. https://doi.org/10.1016/S0734-189X(87)80014-2
  15. Versace M., Kozma R. T., and Wunsch D. C., “Adaptive resonance theory design in mixed memristive-fuzzy hardware”, in Advances in Neuromorphic Memristor Science and Applications, R. Kozma, R. Pino, and G. Pazienza, Eds. Netherlands: Springer, 2012, pp. 133–153. https://doi.org/10.1007/978-94-007-4491-2_9
  16. Xu R. and Wunsch II D., “Survey of clustering algorithms”, IEEE Trans. Neural Networks, vol. 13, no 3, pp. 645–678, May 2005.
  17. Kalouptisidis N., Signal Processing Systems. Theory and Design. New York: Wiley, 1997.
  18. Tank D. W. and Hopfield J. J., “Simple neural optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit”, IEEE Trans. Circuits Syst., vol. 33, no. 5, pp. 533–541, May 1986. https://doi.org/10.1109/TCS.1986.1085953
  19. Xia Y. and Wang J., “A one-layer recurrent neural network for support vector machine learning”, IEEE Trans. Systems, Man and Cybernetics – Part B: Cybernetics, vol. 34, no. 2, pp. 1261–1269, 2004. https://doi.org/10.1109/TSMCB.2003.822955
  20. Carpenter G. A., Grossberg S., and Reynolds J. H., “ARTMAP: supervised real-time  learning and classification of nonstationary data by a self-organizing neural network”, Neural Networks, vol. 4, no 5, pp. 565– 588, Feb. 1991. https://doi.org/10.1016/0893-6080(91)90012-T
  21. Liu L., Huang L., Lai M., and Ma C., “Projective ART with buffers for the high dimensional space clustering and application to discover stock associations”, Neurocomputing, vol. 72, nos. 4-6, pp. 1283–1295, Jan. 2009. https://doi.org/10.1016/j.neucom.2008.01.020
  22. Kim S. and Wunsch II D., “A GPU based parallel hierarchical fuzzy ART clustering”, in Proc. Int. Joint Conf. Neural Networks, 2011, pp. 2778–2782. https://doi.org/10.1109/IJCNN.2011.6033584
  23. Tymoshchuk P. V., “A dynamic K-winners take all analog neural circuit”, in Proc. IVth Int. Conf. “Perspective technologies and methods in MEMS design”, Lviv-Polyana, Ukraine, May 21–24, 2008, pp. 13–18. https://doi.org/10.1109/MEMSTECH.2008.4558724
  24. Tymoshchuk P. V., “A discrete-time dynamic K-winners-take-all neural circuit”, Neurocomputing, vol. 72, nos. 13–15, pp. 3191–3202, Aug. 2009. https://doi.org/10.1016/j.neucom.2009.03.001
  25. Cai X., Prokhorov D. and Wunsch D., “Training winner-take-all simultaneous recurrent neural networks”, IEEE Trans. Neural Networks, vol. 18, no 3, pp. 674–684, May 2007. https://doi.org/10.1109/TNN.2007.891685
  26. Carpenter G. A., Grossberg S., Markuzon N., Reynolds J. H., and Rosen D. B., “Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps”, IEEE Trans. Neural Networks, vol. 3, no. 5, pp. 698–713, Sept. 1992. https://doi.org/10.1109/72.159059
  27. Lopes M. L. M., Minussi C. R., and Lotufo A. D. P., “Electric load forecasting using a fuzzy ART and ARTMAP neural network”, Applied Soft Computing, vol. 5, no. 2, pp. 235–244, Jan. 2005. https://doi.org/10.1016/j.asoc.2004.07.003
  28. Meng L., Tan A.-H., and Xu D., “Semi-supervised heterogeneous fusion for multimedia data co- clustering”, IEEE Trans. Knowledge Data Engineering, vol. 26, no. 9, pp. 2293–2306, 2014. https://doi.org/10.1109/TKDE.2013.47
  29. Carpenter G. A., Grossberg S., and Rosen D. B., “A neural network realization of fuzzy ART”, Technical Report CAS/CNS-91-021. Boston, MA: Boston University, 1991.
  30. Serrano-Gotarredona T., Linares-Barranco B., and Andreou A. G., Adaptive Resonance Theory Microchips: Circuit Design Techniques. Norwell, MA: Kluwer, 1998. https://doi.org/10.1007/978-1-4419-8710-5
  31. Xu R., Xu J., and Wunsch II D. C., “Using Default ARTMAP for cancer classification with MicroRNA expression signatures”, in Proc. Int. Joint Conf. Neural Networks, 2009, pp. 3398–3404. https://doi.org/10.1109/IJCNN.2009.5178603
  32. Xu R., Xu J., and Wunsch II D. C. “MicroRNA expression profile based cancer classification using Default ARTMAP”, Neural Networks, vol. 22, pp. 774–780, June 2009. https://doi.org/10.1016/j.neunet.2009.06.018
  33. Meng L., Tan A.-H., and Wunsch D. C., “Vigilance adaptation in adaptive resonance theory”, in Proc. Int. Joint Conf. Neural Networks, 2013, pp. 1–7. https://doi.org/10.1109/IJCNN.2013.6706857
  34. Tymoshchuk P. V., “A simplified continuous-time model of analogue K-winners-take-all neural circuit”, in Proc. XI Int. Conf. “The Experience of Designing and Application of CAD Systems in Microelectronics”, Polyana- Svalyava, Ukraine, February 23–25, 2011, pp. 121–125.
  35. Cichocki A. and Unbehauen R., Neural networks for optimization and signal processing, Baffins Lane, Chichester: John Wiley & Sons, 1993.