Improving prediction accuracy by artificial intelligence tools is an important task in various industries, economics, medicine. Ensemble learning is one of the possible options to solve this task. In particular, the construction of stacking models based on different machine learning methods, or using different parts of the existing data set demonstrates high prediction accuracy of the. However, the need for proper selection of ensemble members, their optimal parameters, etc., necessitates large time costs for the construction of such models. This paper proposes a slightly different approach to building a simple but effective ensemble method. The authors developed a new model of stacking of nonlinear SGTM neural-like structures, which is based on the use of only one type of ANN as an element base of the ensemble and the use of the same training sample for all members of the ensemble. This approach provides a number of advantages over the procedures for building ensembles based on different machine learning methods, at least in the direction of selecting the optimal parameters for each of them. In our case, a tuple of random hyperparameters for each individual member of the ensemble was used as the basis of ensemble. That is, the training of each combined SGTM neural-like structure with an additional RBF layer, as a separate member of the ensemble occurs using different, randomly selected values of RBF centers and centersfof mass. This provides the necessary variety of ensemble elements. Experimental studies on the effectiveness of the developed ensemble were conducted using a real data set. The task is to predict the amount of health insurance costs based on a number of independent attributes. The optimal number of ensemble members is determined experimentally, which provides the highest prediction accuracy. The results of the work of the developed ensemble are compared with the existing methods of this class. The highest prediction accuracy of the developed ensemble at satisfactory duration of procedure of its training is established.
[1] Agarwal, S., & Chowdary, C. R. (2020). A-Stacking and A-Bagging: Adaptive versions of ensemble learning algorithms for spoof fingerprint detection. Expert Systems with Applications, 146, 113160. https://doi.org/10.1016/j.eswa.2019.113160
[2] Boodhun, N., & Jayabalan, M. (2018). Risk prediction in life insurance industry using supervised learning algorithms. Complex & Intelligent Systems, 4(2), 145–154. https://doi.org/10.1007/s40747-018-0072-1
[3] Chaurasia, V., & Pal, S. (2021). Stacking-Based Ensemble Framework and Feature Selection Technique for the Detection of Breast Cancer. SN Computer Science, 2(2), 67. https://doi.org/10.1007/s42979-021-00465-3
[4] Feng, D.-C., Liu, Z.-T., Wang, X.-D., Chen, Y., Chang, J.-Q., Wei, D.-F., & Jiang, Z.-M. (2020). Machine learning-based compressive strength prediction for concrete: An adaptive boosting approach. Construction and Building Materials, 230, 117000. https://doi.org/10.1016/j.conbuildmat.2019.117000
[5] Folberth, C., Elliott, J., Müller, C., Balkovič, J., Chryssanthacopoulos, J., Izaurralde, R. C., Jones, C. D., Khabarov, N., Liu, W., Reddy, A., Schmid, E., Skalský, R., Yang, H., Arneth, A., Ciais, P., Deryng, D., Lawrence, P. J., Olin, S., Pugh, T. A. M., … Wang, X. (2019). Parameterization-induced uncertainties and impacts of crop management harmonization in a global gridded crop model ensemble. PLOS ONE, 14(9), e0221862. https://doi.org/10.1371/journal.pone.0221862
[6] Hassan, A. H. A., & Elfaki, E. (2018). Prediction of Electrical Output Power of Combined Cycle Power Plant Using Regression ANN Model. https://doi.org/10.5281/zenodo.1285164
[7] Ighalo, J. O., Adeniyi, A. G., & Marques, G. (2020). Application of linear regression algorithm and stochastic gradient descent in a machine – learning environment for predicting biomass higher heating value. Biofuels, Bioproducts and Biorefining, 14(6), 1286–1295. https://doi.org/10.1002/bbb.2140
[8] Izonin, I., Tkachenko, R., Kryvinska, N., Gregus, M., Tkachenko, P., & Vitynskyi, P. (2019). Committee of SGTM Neural-Like Structures with RBF kernel for Insurance Cost Prediction Task. 2019 IEEE 2nd Ukraine Conference on Electrical and Computer Engineering (UKRCON), 1037–1040. https://doi.org/10.1109/UKRCON.2019.8879905
[9] Kurz, C. F., Maier, W., & Rink, C. (2020). A greedy stacking algorithm for model ensembling and domain weighting. BMC Research Notes, 13(1), 1–6. https://doi.org/10.1186/s13104-020-4931-7
[10] Medical Cost Personal Datasets. (n.d.). Retrieved 9 December 2018, from https://www.kaggle.com/mirichoi0218/insurance
[11] Pavlyshenko, B. (2018). Using Stacking Approaches for Machine Learning Models. 2018 IEEE Second International Conference on Data Stream Mining Processing (DSMP), 255–258. https://doi.org/10.1109/DSMP.2018.8478522
[12] Pham, K., Kim, D., Park, S., & Choi, H. (2021). Ensemble learning-based classification models for slope stability analysis. Catena, 196, 104886. https://doi.org/10.1016/j.catena.2020.104886
[13] Rocca, J. (2021, March 21). Ensemble methods: Bagging, boosting and stacking. Medium. https://towardsdatascience.com/ensemble-methods-bagging-boosting-and-stacking-c9214a10a205
[14] Salah, M., Altalla, K., Salah, A., & Abu-Naser, S. S. (2018). Predicting Medical Expenses Using Artificial Neural Network. International Journal of Engineering and Information Systems (IJEAIS), 2(10), 7.
[15] Shaikhina, T., & Khovanova, N. A. (2017). Handling limited datasets with neural networks in medical applications: A small-data approach. Artificial Intelligence in Medicine, 75, 51–63. https://doi.org/10.1016/j.artmed.2016.12.003
[16] Shakhovska, N., Yakovyna, V., & Kryvinska, N. (2020). An Improved Software Defect Prediction Algorithm Using Self-organizing Maps Combined with Hierarchical Clustering and Data Preprocessing. In S. Hartmann, J. Küng, G. Kotsis, A.M. Tjoa, & I. Khalil (Eds.), Database and Expert Systems Applications (pp. 414–424). Springer International Publishing. https://doi.org/10.1007/978-3-030-59003-1_27
[17] Teslyuk, V., Kazarian, A., Kryvinska, N., & Tsmots, I. (2021). Optimal Artificial Neural Network Type Selection Method for Usage in Smart House Systems. Sensors, 21(1), 47. https://doi.org/10.3390/s21010047
[18] Tkachenko, R., & Izonin, I. (2019). Model and Principles for the Implementation of Neural-Like Structures Based on Geometric Data Transformations. In Z. Hu, S. Petoukhov, I. Dychka, & M. He (Eds.). Advances in Computer Science for Engineering and Education (Vol. 754, pp. 578–587). Springer International Publishing. https://doi.org/10.1007/978-3-319-91008-6_58
[19] Tkachenko, R., Izonin, I., Vitynskyi, P., Lotoshynska, N., & Pavlyuk, O. (2018). Development of the Non-Iterative Supervised Learning Predictor Based on the Ito Decomposition and SGTM Neural-Like Structure for Managing Medical Insurance Costs. Data, 3(4), 46. https://doi.org/10.3390/data3040046
[20] Tkachenko, R., Kutucu, H., Izonin, I., Doroshenko, A., & Tsymbal, Y. (n.d.). Non-Iterative Neural-Like Predictor for Solar Energy in Libya. 11.
[21] Tkachenko, R., Tkachenko, P., Izonin, I., Vitynskyi, P., Kryvinska, N., & Tsymbal, Y. (2019). Committee of the Combined RBF-SGTM Neural-Like Structures for Prediction Tasks. In I. Awan, M. Younas, P. Ünal, & M. Aleksy (Eds.). Mobile Web and Intelligent Information Systems (pp. 267–277). Springer International Publishing. https://doi.org/10.1007/978-3-030-27192-3_21
[22] Tsmots, I., & Skorokhoda, O. (2010). Methods and VLSI-structures for neural element implementation. 2010 Proceedings of VIth International Conference on Perspective Technologies and Methods in MEMS Design, 135–135.
[23] Tsmots, I., Skorokhoda, O., & Rabyk, V. (2016). Structure and Software Model of a Parallel-Vertical Multi-Input Adder for FPGA Implementation, 158–160. https://doi.org/10.1109/STC-CSIT.2016.7589894
[24] Tsmots, I., Teslyuk, V., & Vavruk, I. (2013). Hardware and software tools for motion control of mobile robotic system. 2013 12th International Conference on the Experience of Designing and Application of CAD Systems in Microelectronics (CADSM), 368–368.
[25] Xiao, Y., Wu, J., Lin, Z., & Zhao, X. (2018). A deep learning-based multi-model ensemble method for cancer prediction. Computer Methods and Programs in Biomedicine, 153, 1–9. https://doi.org/10.1016/j.cmpb.2017.09.005