An R2 score or a coefficient of determination is used often as a metric to evaluate regression models. It can be applied solely but usually it is combined with other metrics in order to increase accuracy of a model evaluation. The goal of the work is to research the dynamics of the R2 score of a K-Nearest Neighbors regression model trained on series of different sizes in order to propose a new approach to increase the robustness and accuracy of the model evaluation when the R2 score metric is used solely. Typically, a value of the R2 score metric above 0.8 is considered to be sufficient while an evaluated model is considered to be accurate enough. However, such a way of R2 score interpretation to may lead to model’s accuracy misevaluation, which is shown in the proposed paper. The results obtained clearly display that R2 score can vary significantly in some cases depending on the samples selected to test part of a series used for model evaluation. The mentioned variation can contribute to model’s accuracy overestimation, which, in turn can lead to incorrect results of model application. The known methods to make model estimation more accurate involve use of other metrics. Instead, this paper focuses on increase of model’s accuracy estimation without the necessity of using other metrics. The R2 score dynamics is examined using 25000 cycles of the K-Nearest Neighbors regression model training and evaluation. Selection of samples to a training or test part of a series has been done randomly. For all the experiments quantity of neighbors is fixed and equals to the default value of n_neighbors=5 of the KNeighborsRegressor method provided by the Sklearn library. The paper both states and proves a hypothesis that the R2 score variation is expected to increase with series size reduction and the variation is supposed to be observed for models trained on the same series because of training/test samples selection randomness. The experiments carried out allowed to propose an alternative approach that did not require any supplementary metrics. The proposed approach considers application of the R2 score along with its variation that must not exceed 0.2 for the K-Nearest Neighbors regression model.
[1] Sarkar, D., Bali, R., Sharma, T. (2018). Practical Machine Learning with Python. A Problem-Solver's Guide to Building Real-World Intelligent Systems. Apress Berkeley, CA, 545 p. DOI: 10.1007/978-1-4842-3207-1.
[2] Scikit-learn library web page. Sklearn.metrics.r2_score. Available at https://scikit-learn.org/stable/modules/generated/sklearn.metrics.r2_sco...
[3] Nakagawa S., Johnson, P., Schielzeth, H. (2017). The coefficient of determination R2 and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded. Journal of The Royal Society Interface vol. 14(134), pp. 1–11. DOI: 10.1098/rsif.2017.0213.
[4] Zhang, D. (2017) A Coefficient of Determination for Generalized Linear Models, The American Statistician, vol. 71:4, pp. 310–316, DOI: 10.1080/00031305.2016.1256839
[5] Gurubaran, K. et al. (2023). Machine Learning Approach for Soil Nutrient Prediction. 2023 IEEE Silchar Subsection Conference (SILCON), Silchar, India, 2023, pp. 1-6. DOI: 10.1109/SILCON59133.2023.10405095.
[6] Gehlot, A., Sidana, N., Jawale, D., Jain, N., Singh, B.P., Singh,B. (2022). Technical analysis of crop production prediction using Machine Learning and Deep Learning Algorithms. 2022 International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES), Chennai, India, 2022, pp. 1-5. DOI: 10.1109/ICSES55317.2022.9914206.
[7] Tran, T. T. H., et al. (2022). Polygenic risk scores adaptation for Height in a Vietnamese population. 14th International Conference on Knowledge and Systems Engineering (KSE), Nha Trang, Vietnam, 2022, pp. 1-7. DOI: 10.1109/KSE56063.2022.9953620.
[8] Aulia, Y., Purnamasari, P.D., Zulkifli, F.Y. (2023). A Comparative Analysis of Machine Learning Algorithms for Predicting the Dimensions of Rectangular Microstrip Antennas. 2023 IEEE International Symposium On Antennas And Propagation (ISAP), Kuala Lumpur, Malaysia, 2023, pp. 1-2. DOI: 10.1109/ISAP57493.2023.10388517.
[9] Shashank, S., Gourisaria, M.K., Bilgaiyan, S. (2023). Weather Forecasting Based Shared Bike Demand Analysis using Machine Learning. 6th International Conference on Information Systems and Computer Networks (ISCON), Mathura, India, 2023, pp. 1-6. DOI: 10.1109/ISCON57294.2023.10112160.
[10]Kumar, A., Mishra, S.K., Kejriwal, A. (2022). Prediction of Happiness Score of Countries by Considering Maximum Infection Rate of People by COVID-19 using Random Forest Algorithm. 2nd International Conference on Intelligent Technologies (CONIT), Hubli, India, 2022, pp. 1-6. DOI: 10.1109/CONIT55038.2022.9847791.
[11]Géron, A. (2019). Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow. O’Reilly Media, Inc., Sebastopol, CA., USA. 510 p. ISBN: 9781492032649.
[12]Pandas library web page via NumFOCUS Inc. Available at https://pandas.pydata.org/
[13]NumPy. The fundamental package for scientific computing with Python by NumPy team. Available at https://numpy.org/
[14]Scikit-learn library web page. Sklearn.neighbors.KNeighborsRegressor. Available at https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNei...
[15]Matplotlib.pyplot by the Matplotlib development team. Available at https://matplotlib.org/3.5.3/api/_as_gen/matplotlib.pyplot.html