Exploit computer vision inpainting approach to boost deep learning models

In today’s world, the amount of available information grows exponentially every day. Most of this data is visual data. Correspondingly, the demand for the algorithm of image rent is growing. Traditionally, the first approaches to computer vision problems were classical algorithms without the use of machine learning. Such approaches are limited by many factors. First of all, the conditions imposed on the input images are applied – the shooting angle, lighting, position of objects on the scene, etc. Other classical algorithms cannot meet the needs of modern computer vision problems.

Neural network approaches and deep learning models have largely replaced classical programming algorithms. The greatest advantage of deep neural networks in computer vision tasks is not only the possibility of automatically building data processing algorithms that cannot be built in any other way, but also the comprehensiveness of such an approach – actual deep neural networks provide all stages of image processing from start to finish. But. This approach is not always optimal. Training models require a large amount of annotated data to avoid the effect of overfitting such models. In many settings, the conditions have a significant degree of variability, but are limited. In such cases, the combination of both approaches of computer vision is fruitful – pre-processing of the image is performed by classical algorithms, and prediction (classification, object search, etc.) is performed by a neural network.

This article noted an example of the use of damaged images in the classification of tasks (in the extreme cases, the percentage of damage reached 60 % of the image area). We have shown in practice that the use of classic approaches for restoration of damaged areas of the image (inpainting) made it possible to increase the final accuracy of the model by up to 10 % compared to the base model trained under identical conditions on the original data.