Optimization of the Data Labeling Process in Weakly Supervised Environments

The article investigates the problem of increasing the efficiency of the data labeling process in poorly controlled environments based on Active Learning methods. The relevance of the work is due to the rapid growth of unstructured and partially labeled data, the high cost of manual annotation, the shortage of qualified experts, and the negative impact of noise labels on the quality of machine learning models. Traditional approaches to forming training samples do not provide the optimal ratio between the quality of models, resource costs, and time characteristics, which necessitates the development of adaptive mechanisms for controlling the labeling process. A formal model of the annotation process is proposed, which includes the annotation cost function and a budget constraint that allows assessing the efficiency of sample selection. A system architecture is developed, which includes an uncertainty assessment module, a budget management module, an expert interface, a label validation module, and an Active Learning cycle manager. This structure ensures the integration of algorithmic, resource and expert components in a single controlled loop. The method is implemented using a modern technology stack (Python, PyTorch, FastAPI, PostgreSQL, Label Studio, MLflow) and microservice architecture with support for REST API and MLOps pipeline. The proposed approach provides scalability, reproducibility of experiments and the possibility of integration into industrial information systems. The results of experimental studies indicate the effectiveness of the method: an increase in Accuracy and F1-score indicators under limited budget conditions was achieved, labeling costs were reduced compared to random selection and classic Active Learning without budget optimization. It is shown that adaptive budget management and integration of label validation procedures can reduce the negative impact of noisy annotations and increase the economic feasibility of the training process. The results obtained indicate the practical suitability of the developed method for use in weakly supervised information systems and create a basis for further research in the direction of integrating reinforcement learning, multi-agent expert interaction models, and automated weak label generation.