Machine learning models selection under uncertainty: application in cancer prediction

Cancer stands as the foremost global cause of mortality, with millions of new cases diagnosed each year.  Many research papers have discussed the potential benefits of Machine Learning (ML) in cancer prediction, including improved early detection and personalized treatment options.  The literature also highlights the challenges facing the field, such as the need for large and diverse datasets as well as interpretable models with high performance.  The aim of this paper is to suggest a new approach in order to select and assess the generalization performance of ML models in cancer prediction, particularly for datasets with limited size.  The estimates of the generalization performance are generally influenced by numerous factors throughout the process of training and testing.  These factors include the impact of the training–testing ratio as well as the random selection of datasets for training and testing purposes.