CRITERIA FOR THE QUALITY ASSESSMENT OF LARGE LANGUAGE MODELS

The development of large language models (LLMs) with each new iteration demonstrates a significant improvement in their ability to understand and generate text, which opens up increasingly wide opportunities for their integration into information processing systems and digital business processes of enterprises and institutions. In the context of the constant growth of the complexity and functional capabilities of LLMs, the development of reliable methods for their evaluation becomes a fundamental challenge for the research community, as traditional metrics for evaluating text information are often unable to fully cover the entire depth and multifaceted nature of their potential capabilities and characteristics. The creation of comprehensive LLM quality assessment systems is designed to ensure not only an objective comparison of different models but also to form critical feedback for targeted technology improvement and the prevention of potential risks associated with their large-scale implementation. The article is devoted to the systematization of criteria for assessing the quality of large language models that have become widespread in natural language processing tasks. The article aims to create a comprehensive approach to LLM quality assessment that covers the main aspects of their functioning. The paper defines LLM quality criteria such as precision and completeness of responses, naturalness of speech, consistency, toxicity, bias, security vulnerabilities, among others. Three main approaches to assessing LLM quality are analyzed in detail: expert assessments, comparison with reference data, and automated methods without reference data. For each quality criterion, the most effective assessment methods are determined, and their advantages and disadvantages in different application contexts are noted. In conclusion, it is emphasized that, despite the high reliability of expert assessments, automated methods are becoming increasingly important for large-scale LLM evaluation, especially for subjective criteria such as toxicity or bias