Mathematical modeling of multi-label classification of job descriptions using transformer-based neural networks

This article presents the mathematical modeling of the multi-label classification task of job description texts aimed at the automatic detection of working conditions and social benefits, which can enhance communication efficiency between employers and job seekers.  The proposed approach is based on the use of the transformer-based BERT neural network, pre-trained on a multilingual corpus.  The dataset was constructed by collecting job postings from the three largest Ukrainian job search platforms: Work.ua, Robota.ua, and Jooble.org.  The collected texts were augmented with artificially generated examples using large language models to ensure class balance.  An architecture was implemented for fine-tuning the BERT model in a multi-label classification mode using the Binary Cross-Entropy loss function.  To determine the optimal training configuration, a comparative analysis of four popular optimizers (SGD, AdaGrad, RMSprop, AdamW) was conducted under various learning rate values.  The model's performance was evaluated using precision, recall, and F1-score metrics. The experimental results demonstrated that the highest classification quality was achieved using the AdamW optimizer with an appropriately selected learning rate.  The novelty of the study lies in combining transformer architecture with an applied task in the field of job description text processing, which enables increased informativeness of postings and automation of preliminary analysis of working conditions.  The proposed approach can serve as a foundation for developing tools in HR systems and can be integrated into recruitment platforms to improve the relevance of job postings to the needs of target audiences.