DEVISING A METHOD FOR IDENTIFYING THE MODEL OF MULTI-CRITERIA EXPERT ESTIMATION OF ALTERNATIVES

An intelligent decision-making process [1, 2] implies implementation by a decision-maker (DM) of a procedure of choosing the “best” alternative (or ranking of alternatives) from some available limited set. In turn, this problem can be divided into two subproblems. The first of them relates to substantiation and selection of a metric (a system of criterion estimates) in which the effectiveness (usefulness) of alternatives can be qualitatively or quantitatively estimated. The second subproblem implies choosing the allowable extremal solution in this metric. In cases where the problem facing the DM is quite difficult to solve, competent specialists (experts) who are well versed in this subject area should be involved. Generally speaking, the set of judgments obtained from the experts, methods of forming quantitative and qualitative estimates as well as the processing of results using formal methods provide a basis of the expert estimation method [1, 3]. Sense of the expert’s activity implies the estimation of effectiveness of each alternative in general based on analysis of information on a set of contradictory criteria that characterize its specific partial properties. The analysis complexity also lies in the fact that these partial criteria can be measured using various scales, they have various ranges of possible values, dimensionalities, directions of dominance, and degrees of importance. This problem is known in the How to Cite: Petrov, K., Kobzev, I., Orlov, O., Kosenko, V., Kosenko, A., Vanina, Y. (2021). Devising a method for identifying


Introduction
An intelligent decision-making process [1,2] implies implementation by a decision-maker (DM) of a procedure of choosing the "best" alternative (or ranking of alternatives) from some available limited set. In turn, this problem can be divided into two subproblems. The first of them relates to substantiation and selection of a metric (a system of criterion estimates) in which the effectiveness (usefulness) of alternatives can be qualitatively or quantitatively estimated. The second subproblem implies choosing the allowable extremal solution in this metric.
In cases where the problem facing the DM is quite difficult to solve, competent specialists (experts) who are well versed

A l i s a K o s e n k o
PhD, Associate Professor** Y a n a V a n i n a PhD, Аssociate Professor** *Department of Digital Technologies and Electronic Government*** **Department of Economic Policy and Management ***Educational and Scientific Institute "Institute of Public Administration" of Kharkiv National University named after V. N. Karazina Moskovskyi ave., 75, Kharkiv, Ukraine, 61050

An approach to constructing mathematical models of individual multicriterial estimation was proposed based on information about the ordering relations established by the expert for a set of alternatives. Structural identification of the estimation model using the additive utility function of alternatives was performed within axiomatics of the multi-attribute utility theory (MAUT). A method of parametric identification of the model based on the ideas of the theory of comparative identification has been developed. To determine the model parameters, it was proposed to use the midpoint method that has resulted in the possibility of obtaining a uniform stable solution of the problem. It was shown that in this case, the problem of parametric identification of the estimation model can be reduced to a standard linear programming problem. The scalar multicriterial estimates of alternatives obtained on the basis of the synthesized mathematical model make it possible to compare them among themselves according to the degree of efficiency and, thus, choose "the best" or rank them. A significant advantage of the proposed approach is the ability to use only non-numerical information about the decisions already made by experts to solve the problem of identifying the model parameters. This enables partial reduction of the degree of expert's subjective influence on the outcome of decision-making and reduces the cost of the expert estimation process. A method of verification of the estimation model based on the principles of cross-valida-
decision-making theory as the problem of multicriteria estimation.
The next task that experts are facing implies choosing an extreme solution (an alternative). This is the so-called multicriteria optimization problem. The main difficulty that arises when solving such problems implies the fact that choice of an alternative is often made using a subset of acceptable solutions known as a compromise area or Pareto set with which it is impossible to establish a linear relationship (rank alternatives), and, consequently, identify an extreme alternative (the only solution). This is caused by the fact that alternatives are incomparable because of the inconsistency of partial criteria. Thus, the problem of multicriteria optimization is mathematically incorrect in such a statement [4] because there is no uniform solution. To define it, it is necessary to regularize this problem by supplementing it with external information obtained from experts which would be formalized in a form of a rule.
The theory of rational behavior assuming that an individual chooses the alternative that gives the best result (the most effective one) is the basis of a constructive approach to solving the general decision-making problem. In turn, estimation of the effectiveness of the alternative can be obtained based on the use of ideas of the multi-attribute utility theory (MAUT) [5]. It assumes that there is some function of the alternative usefulness which depends on partial criteria. After calculating the value of the utility function for each of the alternatives, the "best" one can be chosen or rank the alternatives based on analysis of these values.
Thus, the approach to the MAUT-based decision-making process formally implies assigning to the alternatives some generalized quantitative estimates of their usefulness. The purpose of this process implies enabling the expert to select later the "best" of them or rank them according to the degree of personal preference. Therefore, the current problem implies formalization of approaches to the formation of these generalized estimates of alternatives based on expert judgments.
The process of constructing a mathematical model of individual multicriterial estimation of alternatives can be represented as a sequence of two stages. The first of them implies hypothesizing the model structure (the problem of structural identification) and the second implies determining its parameters (the problem of parametric identification).
The existence of a multidimensional function of utility specified for the set of alternatives under consideration was proved within the framework of the MAUT axiomatics. Therefore, the problem of structural identification of the estimation model within the MAUT frames was actually solved. Either an additive or a multiplicative form of recording the utility function is used when solving practical problems.
The solution to the problem of parametric identifying the model is of high interest.

Literature review and problem statement
The model parameters are most often determined in a series of active or passive experiments conducted in cooperation with experts. This results in the fact that their judgments are usually implemented in a form of direct estimates, matrices of paired or multiple comparisons, various ranking and classification procedures [3].
Direct estimation, in this case, will imply attributing some numerical values measured in the interval scale to the partial criteria characterizing the alternative. These values correspond to the degree of influence of one or another criterion on the result, i.e., multicriterial estimation of the alternative.
When identifying the knowledge, the expert must set up exact values on a continuous numerical axis (e.g. in a segment [0, 1]) in accordance with each criterion. The same values should be attributed to the same "important" criteria. This is a natural requirement in this case.
In some cases, to weaken these conditions, scores (such as 5, 10, 100-point scales) are used instead of continuous numerical axis but naturally at the expense of reduction of measurement accuracy.
This approach to the estimation of the "importance" of criteria (in one form or another) is used in various wellknown decision-making methods. For example, conditions and principles of using the ELECTRE method and the problems that arise when obtaining expert's information on the criteria "importance" are considered in detail in [6]. Study [7] addresses the description of the PROMETH-EE method. The authors note that when implementing this method, difficulties occur when experts are setting "weights" of criteria and functions of alternative's advantages. Applicability of the TOPSIS method and the difficulties arising in formalizing the expert's judgments on the choice of an "ideal" alternative and assigning "weights" to the criteria are described in [8]. It is pointed out in [9] that the requirement for experts to determine quantitative values of the criteria "importance" is the main problem in obtaining a scalar multicriterial estimate of alternatives by SMART, SMARTS, and SMARTER methods.
Numerous publications point to the popularity of the use of these methods. The author of [10] notes that the requirement of independence of criteria (although they may be interrelated in real problems) is a significant limitation of the ELECTRE method. Study [11] addresses solving the problem of selection of product suppliers. "Weights" of the criteria are determined using the mathematical apparatus of the Dempster-Schaefer theory of evidence. Study [12] addresses modeling uncertainty in the problem of assigning advantages to alternatives by a DM using the Monte Carlo method within the framework of the PROMETHEE approach. Study [13] compares results of solving the problem of deciding on student tuition fees by using four different methods. The authors also point out the difficulties associated with obtaining quantitative information about the criteria "weight". The problem of interpersonal influences on collective decision-making with the use of TOPSIS is investigated in [14]. To solve this problem, it is necessary to obtain from experts estimates of all alternatives for all criteria (the socalled decision matrix).
SMART and SMARTER methods are often used to solve practical decision-making problems. For example, the problem of estimating the level of staff discipline is solved in [15], the problem of estimation and ranking students based on their educational achievements is discussed in [16]. Study [17] addresses the problem of deciding on the use of alternative strategies in coping with Dengue fever. When applying these methods, experts must directly quantify the criteria "weights" which often causes difficulties for them. Therefore, we can conclude that the main disadvantage of this approach implies the fact that benefits can be mea-sured in the interval scale with a high degree of confidence only at good awareness of the expert about characteristics of alternatives and the subject area itself.
The method of pairwise comparisons implies implementing the procedure of establishing the criteria advantages when comparing all possible pairs. When comparing each pair of criteria, relations of either order or equivalence are possible. Pairwise comparison is a measurement in the order scale.
Results of the expert's comparison of all pairs of criteria are usually presented in the form of a matrix of pairwise comparisons in which columns and rows make up the criteria and its corresponding elements are numerical values of their "importance". Comparison of criteria in all possible pairs does not give a complete ordering of all criteria. This is the main disadvantage of the method.
The hierarchy analysis procedure (HAP) and its development, the analytical network procedure (ANP) [18], are the most commonly used current approaches applying the method of pairwise comparisons.
Numerous publications show the popularity of using this approach in practice. For example, the NAP is used in [19] to evaluate and compare the quality of yarn and the problem of choosing an all-terrain vehicle model for equipping military units is solved in [20]. Study [21] addresses the development of a method for evaluating employee work effectiveness in order to rank them based on the ANP application. The issues of estimating the safety of alternative construction projects and determining the priority of potential risks in the construction sector are considered in [22].
Solution of all these problems requires information from experts in the form of matrices of pairwise comparison of the importance of criteria and alternatives while fulfilling conditions of consistency of the obtained matrices which causes some difficulties to the experts. This and other problems related to conflicting priorities and attitudes, conflicting solutions when applying the HAP method in practice are analyzed in detail in [23].
Ranking is a procedure of sorting the criteria according to the degree of their influence on the result. Based on his knowledge and experience, the expert arranges criteria in an order of their "importance" guided by one or more comparison indicators. Depending on the type of relationship between objects, there are various options for organizing the criteria.
Ranking makes it possible to choose the most significant criterion from the set of criteria under study.
A sequence of natural numbers called ranks is most often used for ranking in the practice of expert estimation. Rank 1 is assigned to the most "important" criterion, rank 2 is assigned to the next less "important" criterion, etc.
The most popular methods that use this approach include methods of the main criterion [2], optimization according to sequentially applied criteria, and successive deductions [24].
Despite the presence of numerous methods for obtaining information from experts, all of them are subjective to one degree or another. It relates to the need to take into account and formalize expert judgments on the "importance" of the criteria characterizing the alternatives. Therefore, it is necessary to develop methods that would reduce the degree of this subjective influence of experts on the outcome of multi-criteria estimation of alternatives in order to improve the effectiveness of the decisions made.
In this situation, a solution may involve solving the problem of determining parameters of the estimation model (the criteria "weight" coefficients) based on information about the DM decisions or ranking of alternatives.

The aim and objectives of the study
The study objective implied synthesizing a mathematical model of individual multicriterial estimation of alternatives based on available information about the expert's preferences which would fill in the missing information and predict the expert's judgments.
To achieve this objective, it was necessary to solve the following main tasks: to develop methods of structural and parametric identification of the mathematical model of individual multicriterial estimation of alternatives in the framework of MAUT axiomatics based on the ideas of the theory of comparative identification [25]; to verify the constructed mathematical model of estimation in terms of its predictive properties in conditions where there is a small amount of source information obtained from experts.

The study materials and methods
Let the expert is given a finite set of alternatives (the problem solution options) X={x 1 , x 2 ,…,x n }, each described by the same set of partial criteria K( It is assumed that these partial criteria can be measured in quantitative scales and the set X consists only of non-dominant alternatives. In a situation where the criteria are measured in qualitative scales, it is possible to make a transition to quantitative indicators using the Harrington scale. Based on a detailed analysis of these alternatives, the expert should evaluate them in terms of efficiency (usefulness) for a further selection of the "best" one or implement the procedure of their ranking (partial or complete) on the whole set or any subset thereof.
In this case, the situation when the expert has chosen the only most preferred alternative x s ÎX, 1, , s n = can be formally represented as s¹i. Ranking of alternatives results in establishing linear order on the set X or on any of its subsets or a relation, for example, x 1 ý(≥)x 2 ý(≥)…ý(≥)x n or a partial linear order, for example, Here "ý" "("≥") and "~" are signs of relations of strict (non-strict) superiority and equivalence, respectively.
The problem implies the construction of a model of multi-criterion estimation of alternatives by an expert within the MAUT in the case when information received from him is only on the selection of the "best" or on establishing the relationship of order on a set of alternatives.
The solution to this problem will make it possible to obtain relative generalized scalar estimates of alternatives in the form of values of functions of their usefulness. It will also enable obtaining estimates for those alternatives for which there are no (for any reason) expert judgments or estimates for "new" alternatives that have not been submitted to the expert for consideration. This will make it possible to fill in the missing information on the benefits of alternatives on the whole set X for each of the experts. This will result in the possibility of using standard methods of estimating con-sistency of their judgments, as well as obtain a consolidated estimate of alternatives or their ranking in the collective expert estimation procedure [26].
The problem of constructing a mathematical model of individual multicriterial expert estimation can be represented as a sequence of two interrelated subproblems. The first implies forming the model structure and the second identifies its parameters based on information received from experts.
Let us consider in detail the process of solving the problem of structural identification of the estimation model.
Based on introspective analysis of partial criteria K( in terms of its effectiveness (usefulness) for a further selection of the most preferred one or makes ranking of alternatives in the decision-making process.
Such an estimation can be performed within the MAUT framework which assumes that for each of the alternatives x i ÎX, there is some scalar multicriterial estimate of generalized utility, P( which has an axiomatic justification [1,27]. It can be used to evaluate alternatives while taking into account all partial criteria. This means that some conditions (axioms) are put forward [1,27] which must be satisfied by the utility function P(x i ) and if they are fulfilled, the existence of P(x i ) as a polylinear function is proved. All axiom conditions can be divided into two groups. The first group includes the following axioms: Archimedean, connectivity, transitivity, solubility. Their implementation allows us to conclude that the system of advantages of the DM expressed in the language of binary relations can be reduced to a problem of single-criterion scalar optimization [1]. The second group consists of the conditions of independence of the criteria implementation which makes it possible to determine properties of the utility function.
Thus, it was proved that under the conditions of the first and second groups of axioms, there is a multidimensional utility function given on a set of alternatives X which can be represented as in [1,27]: Here 0<a i <1, 1, j m = are partial scaling parameters (isomorphism coefficients) which actually show the expert relative importance ("weight") of partial criteria k j (x i ), 1, , j m = which characterize the alternatives x i ÎX; ( ) the single-criterion utility function which characterizes estimation of the alternative x i ÎX by the partial criterion k j (x i ) and satisfies the rationing conditions The general scaling constant w>-1 can be found from the equa- and when 1 1 m j j a = ¹ ∑ , it takes a multiplicative form: When w=0, then based on (1), the multiplicative utility function (3) is reduced to the additive one (2).
According to the MAUT axiomatics for the utility function P(x i ), the following relations are fulfilled when comparing the alternatives: if alternative x i is mainly for the expert than x j , i.e.
if alternatives x i and x j are equivalent, i.e., x i~xj , then For the purposes of this study, the additive form of representation of the utility function of alternative (2) is more suitable. First, there is no need to find the constant w, and secondly, the condition 1 1 m j j a = = ∑ makes it possible to determine relative "weight" coefficients of partial criteria in the usual scale from 0 to 1. This clearly shows the expert how many (or how many times) one partial criterion is "more important" than the other and the "contribution" of each criterion to the overall estimation of the alternative. The multiplicative function (3) does not give any advantages when using it.
Thus, in the future, values of parameters a j , 1, j m = and multicriterial estimates of alternatives P(x i ), 1, i n = will be determined proceeding from the additive form (2) of the utility function representation.
In the general case, partial criteria K(x i )=<k 1 (x i ), k 2 (x i ), …, k m (x i )> are heterogeneous and therefore have different physical dimensions, mismatched measurement intervals, and direction of dominance.
To construct the utility functions for each partial criterion and normalize their values, the following formula will be used [25,26]: where ( ) j i k x is the real (absolute) value of the j-th partial criterion; and ( ) j i k x − is respectively its "worst" or "best" value depending on the direction of the criterion dominance; α j is the coefficient of nonlinearity (α j >0) which allows us to realize, in addition to conventional linear (α z =1), nonlinear, convex up (0<α j <1) or down (α j >1) dependences.
At the same time, H j i k x Î and the "worse" absolute value of the partial criterion will correspond to its smaller value.
Taking into account the accepted method of normalization of partial criteria, the coefficients A=<a 1 , a 2 ,…, a m > become dimensionless and perform two important functions. First, they scale the scalar multicriterial estimation of the alternative P(x i ), i.e. determine the range of its possible values and take into account the different "importance" ("weight") of the partial criteria.
In order that values of P(x i ) vary in the interval [0, 1], it is sufficient that the following conditions are met: Based on this, there are grounds to move to the solution of the problem of parametric identification of the mathemat-ical model of individual multicriterial expert estimation. Its structure is described by the additive function (2).
It is necessary to determine values of the "weight" coefficients of tuple A of the partial criteria K(x i ) based on information only on the choice of the most preferred or on the partial ordering of alternatives. These data can be obtained from the expert in the process of analyzing the set of alternatives presented to him for consideration.
The process of parametric identification of the model (2) is considered below using the example.
Let the expert has formulated his own advantages on a set of four alternatives as follows: Then, for this situation, taking into account (2), (4), and (5), we can write a system of linear constraints which takes the following form: The system of linear constraints (8) distinguishes a convex polyhedron in the hyperplane (7). Any point lying inside this polyhedron is a valid solution. Thus, according to Tikhonov's definition [4], the problem of determining the coefficients of relative "importance" of partial criteria is incorrect because it does not have a uniform solution in this form. To obtain a uniform solution, it is necessary to supplement the original problem with some regularizing relations [4].
There is no additional information that would allow us to put forward an objective hypothesis about the form of this regularizing expression. Therefore, we will proceed from the heuristic assumption that the point estimation of parameters of the multicriterial estimation model should be in the central region of the polyhedron of admissible values of the coefficients A which are determined from the system of constraints (7) and (8). The argument in favor of choosing such a solution may be that it will increase the stability of the estimation model in terms of possible changes in the boundaries of the polyhedron with the receipt of new data.
The paper offers to use the midpoint method to determine a uniform solution to the problem of parametric identification of the model of multicriterial estimation. Let us consider its implementation using the example (8).
First, the point limit values of the admissible set for each of the parameters a j are determined. To do this, the system of constraints (8) is transformed as follows: In the obtained system (9), the signs ">" are changed to "≥" and the equality P(x 1 )−P(x 4 )=0 is represented as two inequalities P(x 1 )−P(x 4 )≥0 and −P(x 1 )+P(x 4 )≥0. Replacement of the constraint signs will not affect the problem solution because it is in the inner region of the polyhedron described by the system (9) and not on its boundaries.
The process of formalization of constraints (8) considering (2) is as follows: where a j , 1, j m = satisfy the conditions (7). It should be noted that a j Î[0, 1], 1, , j m = in conditions (7) although proceeding from MAUT axiomatics, a j Î(0, 1). Softening of the axiom conditions, in this case, is admissible since values of a j will be in the center of the polyhedron of solutions. They will almost never be equal to the extreme values 0 and 1, except for a few isolated cases that will not occur when solving practical problems. Next, the system of constraints (7), (9) is sequentially supplemented by regularizing objective functions of the form: Each of the formulated 2m problems is a problem of linear programming (LP) with an objective function of the form (12) or (13) and a system of constraints (7), (9).
Solution of these problems, e.g. by the simplex method makes it possible to obtain a tuple of interval values of the model parameters a j , i.e.
where min , j a max j a are the limits of the range of allowable a j values determined proceeding from (7), (9), (12), or (13). Thus, as a result of applying the midpoint method, point values of the tuple of the "weight" coefficients of partial criteria A=<a 1 , a 2 ,…,a m > are obtained. This will allow us, firstly, to synthesize a mathematical model of individual multicriterial estimation, and secondly, determine relative scalar multicriterial estimates of alternatives P(x i ), 1, , i n = calculated on the basis of A proceeding from (2). Based on the values of these estimates, we can arrange alternatives according to the degree of their preference for the expert, i.e. rank in ascending or descending order of importance for decision making.
Naturally, the proposed method of regularization of the problem (7), (9) is not the only possible one. For example, [26] proposes a regularization method based on the calculation of the Chebyshev point, and [28] uses a genetic algorithm for this purpose.
To verify the proposed methods of structural and parametric identification of the estimation model, we shall use a technique called cross-validation [29]. It is often used with a small amount of input data in computerized learning methods. Cross-validation in this case will be used to test the method on the same data set. When cross-validation is used, data are divided into learning and test parts. To implement the method, all but one of the parts are used and its testing is performed on the remaining part of the data. These parts can be changed later several times so that the method is implemented and tested on the entire data set. The purpose of cross-validation implies testing the model's ability to predict new data that were not used in its identification.

1. Solving the problem of structural and parametric identification of the estimation model
To demonstrate the operability of the proposed methods of structural and parametric identification of the model of multicriterial estimation, we shall consider the following abstract example.
The expert is asked to consider a set of acceptable solutions consisting of ten non-dominant alternatives each characterized by five partial criteria.
Let us generate values of normalized partial criteria ( ) ( ) , of alternatives XÎ{x i }, 1,10 i = using a sensor of random numbers. All these data are presented in Table 1.
Besides, we shall randomly generate a sequence (ranks) of alternatives according to the degree of reduction of their usefulness to the expert (Table 1, column R*). Thus, the following linear ordering relation is obtained on the set of alternatives: The multicriterial utility function of alternatives Р(х i ), 1,10, i = expressed in the additive form (2) takes the following form: where a j , 1, 5 j = satisfy conditions (6). According to (2), (4), and (12), the following system of linear constraints is obtained: .
Next, it is necessary to make sure that there is a function of the form (18) that "describes" ("approximates") the relation of order (17).
To do this, it is necessary to find the midpoint for the system of constraints (19), i.e. the value * * , j A a = 1, 5, j = using the method described above (14)-(16). All calculations were performed using the Mathcad v.14.0 software (developer: PTC Corporation, USA).
As a result, a model of individual multicriterial estimation of alternatives is obtained in accordance with (13) and (14). It takes the form: Thus, the function P * (x i ) exists which "approximates" the relation of order (17).
Based on (20), we can calculate the relative scalar multicriterial estimates of alternatives (column P*(x i ) in Table 1).

2. Verification of the constructed mathematical model of multicriterial estimation
To verify the synthesized estimation model and methods of its structural and parametric identification, it is necessary to use the cross-validation method [29].
Let us divide the set of alternatives XÎ{x i }, 1,10 i = into learning and test subsets. Let the learning subset contain eight alternatives and the test subset contain two alternatives.
To estimate compliance of the alternative rankings obtained during the experiments with respect to the original R*, we shall use the Spearman rank correlation coefficient [29]: where * i r and qi r are the ranks (places) in the initial relation of ordering the alternatives R* and in the corresponding ordering relations established during the experiments E1, E2, …, Eq where n is the number of alternatives x i ÎX.
Based on the information about ordering relations E1, E2, …, E5 which were set on the learning subsets of alternatives and using the midpoint method to determine parameters of the model of multicriterial estimation of the usefulness of alternatives (18), the following is obtained: for E1: Next, let us consider for comparison a situation where the expert chose only the "best" alternative. In our case, taking into account (17), this means that from the expert's point of view, the alternative x 8 has an advantage over others, i.e. there is the following information: E6: x 8 ý{x 9 , x 4 , x 5 , x 7 , x 6 , x 3 , x 1 , x 2 , x 10 } and the following is obtained based on this information: Values of relative multicriterial estimates (utility functions) are presented in Table 2 for each of the alternatives x i ÎX, 1,10 i = (including those for the testing subset) that were calculated proceeding from P 1 (x i ), P 2 (x i ), …, P 6 (x i ). Table 2 Values of relative multicriterial estimates of alternatives The corresponding ranks of alternatives R 1 , R 2 , …, R 6 determined on the basis of values of their utility functions (Table 2) are presented in Table 3.
Also, Table 3 presents values of the coefficients ρ q , 1, 6, q = which show deviation of the obtained rankings R 1 , R 2 …, R 6 from the original ratio of the order of alternatives R* with a complete coincidence of the two rankings ρ q =1.
Judging from the above, there are grounds to discuss and analyze the results obtained during the computation experiment to verify the proposed methods of structural and parametric identification of the estimation model. The problem of multicriteria estimation of alternative solutions of the problem is non-trivial because of the inconsistency of the criteria characterizing the alternatives. However, it is a key problem for the decision-making process as the choice of a final decision is made on the basis of the obtained estimates of alternatives.
The approach to constructing a model of individual multicriterial estimation based on information about the decisions already made by the expert was considered. Thus, a DM does not require an explicit indication of quantitative values of the criteria of "weight" coefficients as is usually required in most methods used currently. Identification of parameters of the estimation model is based on the information obtainable in two ways. First, it is based on the information obtained in observation of the expert (passive experiment) when he is choosing the "best" alternative. Second, during an active experiment when the expert ranks all or a part of the alternatives in an order of predominance.
The feasibility of such an approach was evidenced by numerous studies. Their results have shown that a person performs the operation of comparison more accurately than that of quantitative measurement (attribution of specific numerical estimates).
The use of the proposed approach to constructing a model of multicriterial estimation of alternatives allows us to solve a series of problems: 1) determine values of relative "weights" of the criteria in the interval and point expression (20) to (25), i.e., the degree of their impact on the generalized estimation of alternatives; 2) calculate quantitative estimates of alternatives which will allow us to switch from the ranking scale to the scale of relations to measure the usefulness of an alternative (Table 2); 3) obtain quantitative estimates of alternatives or their ranks for those that were not considered for some reason by the expert or "new" alternatives, i.e. to "approximate" the missing data on the preference of alternatives (Tables 2, 3).
Because of the limited amount of data to verify the model, the method of cross-validation was used [29].
Proceeding from the obtained values of the Spearman coefficients ρ q (Table 3), the following conclusions can be drawn. During the experiment E1, there were two inversions of the order (ranking) of alternatives R 1 by a decrease of their "usefulness" relative to the original ranking R* (ρ 1 =0.98). There was one inversion (ρ 2 =ρ 3 =ρ 4 =0.99) in experiments E2−E4 and no inversion in experiment E5 (ρ 5 =1.00).
This indicates a high degree of adequacy of the proposed model of individual multicriterial estimation of alternatives.
Quite a large number of inversions (ρ 6 =0.89) in the experiment E6 is explained by a complete lack of information about the ranking of alternatives (there are only data on the "best" alternative). However, even in such a situation, it can be concluded that it is appropriate to build an estimation model to predict the ordering relations established for the whole set of alternatives proposed by the expert for consideration. This case is especially valuable because it does not require conducting a series of active experiments with experts (polling, questioning, etc.) to obtain information about its benefits. Here, the process of obtaining information occurs naturally during observation of an expert's behavior (the so-called passive experiment).
One of the main advantages of the proposed approach is in its ability to obtain "predictive" quantitative multicriterial estimates of alternatives based on information about the decisions already made by the DM and conduct their further ranking based on them. For example, in the absence of complete information on the ranks of alternatives, it is impossible to apply conventional methods of verifying the consistency of expert judgments for each expert. Also, in this case, it will be problematic to use methods of processing the voting results (methods of Bord, Condorcet, Kemeni medians, and ranks) [1,3] which are often used in the procedures of collective expert estimation [26].
Besides, interval information about the "weight" coefficients of partial criteria (the model parameters) A= = that characterize the alternatives can be used to analyze the model sensitivity and calculate interval multicriterial estimates of alternatives as it was done, e.g. in [30].
It is expedient to use the method of parametric identification of the estimation model in a situation where the amount of initial data is relatively small which often characterizes the process of expert estimation in decision making.
Besides, classical methods of approximation are inapplicable in this case because of the fact that the model output contains only non-numerical information about the ordering relation established for a set of alternatives.
Potentials of using the mathematical apparatus of artificial neural networks to solve such problems will also be limited: firstly, because of lack of the necessary amount of input data applicable for their learning, and secondly, this apparatus is poorly adapted to solving the problems of ordinary classification.
Disadvantages and main limitations of the use of the proposed approach consist in the problem of obtaining objective information about the benefits of experts. As mentioned above, such information can only be obtained by conducting a series of active experiments with an expert. However, any intrusion of a stranger in the intelligent process of forming expert judgments affects, one way or another, the outcome of decision-making and leads to "subjectivation" of expert opinions. Construction of a mathematical model of estimation solely based on information about the expert's choice of only the "best" alternative (passive experiment) leads to a decline in the level of the model adequacy. This is explained by a small amount of information used to solve the problem of finding the model parameters.
Prospects for further studies involve comprehensive testing of constructed estimation models when solving various practical problems. Elaboration of hybrid decision-making methods that use a variety of numerical and non-numerical information about expert benefits is also promising. These methods should combine the "best features" of currently applied approaches (including those presented in this paper). Examples of creating such hybrid methods are described in [12][13][14]19].

Conclusions
1. Structure of a model of multicriterial estimation based on the MAUT axiomatics presentable as an additive function of the usefulness of alternatives was substantiated. A midpoint method for parametric model identification has been developed. It is based on the ideas of the theory of comparative identification. In contrast to the existing approaches, the model parameters (the partial criteria "weight") are determined on the basis of information about the DM decisions. In the methods used today, "weights" are postulated in advance by the expert based on the knowledge obtained from him using methods of direct estimation, construction of matrices of pairwise comparisons, or ranking the partial criteria by their "importance". This difference in approaches partially reduces the subjective influence of experts on the estimation result. As a result of applying this approach, the model parameters can be obtained in a form of both interval and point values which in turn makes it possible to calculate generalized scalar quantitative multicriterial estimates of alternatives. This makes it possible to get a stable solution to the problem of choosing the "best" or ranking the alternatives according to the degree of their preference for decision-making. The obtained model of individual multicriterial estimation enables the prediction of values of the functions of alternative utility that were not presented to experts for some reason or "new" alternatives. This substantially reduces the examination cost due to the fact that no additional involvement of experts is required in this case.
2. A method of verifying the estimation model based on the principles of cross-validation has been developed. The Spearman coefficient was used to check the correspondence between the original R* ranking and the R 1 -R 6 rankings obtained in the process of computer simulation. Analysis of its values (ρ 6 =0.89 in the worst case and ρ 5 =1.00 in the best case) allows us to conclude about the adequacy of the constructed model of multicriterial estimation.
It is expedient to use the synthesized estimation model in solving a wide range of problems related to automation of the intelligent process of decision making. The approach proposed in the study can be successfully used, for example, to solve problems of quality estimation of various project decisions, investment management, strategic planning, development of problem-oriented systems of decision support.