Development of Methodological Support for Improving the Quality of Expert Assessment of Business Processes

The object of research is the process of forming a collective expert assessment with increased reliability in making management decisions in business structures by an expanded team of experts. One of the most problematic places in the expert assessment of management decisions is the complexity of forming a competent expert team and the rather high cost of the expertise. In recent years, there has been a tendency for expert assessment with an expanded team of experts. In this case, not only professional experts are involved in the examination, but also all persons wishing to take part in solving the problem. In this case, the reliability of the examination raises doubts. In connection with the participation in expert assessment of persons who do not have experience in expert work, a wide range of expert assessments is possible. The analysis of the current state of the methods of expert assessment in business is carried out. It has been established that the Delphi method, which was most used until recently, does not meet modern requirements. More progressive methods are based on mathematical consensus theory. Consensus is understood as the degree of correlation of individual expert assessments performed in rank scales. In the course of the study, formalized mathematical approaches to the organization of collective expertise were used. A method for processing the results of an examination with an expanded composition of experts was developed. The developed methodology is focused on identifying experts with insufficient qualifications. The methodology allows for a step-by-step assessment of the reliability of the collective expert decision by assessing the Kendall concordance coefficient. It is shown that the phased exclusion of assessments by experts with insufficient qualifications allows increasing the level of consensus, the quality and reliability of the collective expert assessment. The developed methodology has been tested in a really functioning enterprise to make a decision on the exit strategy of the enterprise from their crisis. The use of the developed methodology has made it possible to significantly increase the reliability of the examination results, assessed by the concordance coefficient. The results are useful for practical application in business structures when conducting expert examinations involving a wide range of participants.


Introduction
In the modern world, due to the acceleration of the innovation process, the intellectualization of business process management is increasing. Accordingly, the role of knowledge and experience of experts and consultants who are involved in the expert assessment of various aspects of the enterprise's activities -management, marketing, production, etc., increases. To ensure the competitiveness of enterprises, the intellectual aspect of supporting management decisions is especially important. Making management decisions is one of the key components of a modern business process. In the conditions of the realities of the modern economy, the pace of decision-making is growing rapidly, and information about the state and course of a complex business process is always incomplete. Under these conditions, a traditionally successful method of supporting decision-making is collective expert assessment or expertise -the choice of the optimal solution from the available set of options. Expert assessment is carried out with the involvement of a team of experts, high-class specialists in a specific business area. Recently, within the framework of the concepts of collective intelligence and «wisdom of the crowd», a new direction of expert assessment has been formed -social expertise or expertise with an expanded team of experts [1]. Everyone who wants to contribute to solving the problem takes part in such an examination as experts. In what follows, such an examination will be called an examination with an expanded composition of experts. Expertise with an expanded team of experts, as a procedure for collective assessment of options, emerged in social networks [2,3]. Nevertheless, there is information that the use of peer review with the involvement of a wide range of non-professional experts is also effective for making decisions in business structures. Since in such an examination there can be a very wide range of individual assessments, it is a serious task to obtain a high-ISSN 2664-9969 quality and reliable aggregated expert solution. This task for the expanded team of experts is unsolved and quite urgent.

The object of research and its technological audit
The object of research is the process of forming a collective expert assessment with increased reliability in making management decisions in business structures by an expanded team of experts.
One of the most problematic areas in expert support for management decisions is the complexity of forming a competent expert team and the rather high cost of expertise. In recent years, there has been a tendency to form a collective expert assessment with an expanded team of experts. In this case, not only professional experts are involved in the examination, but also all persons wishing to take part in solving the problem. In such situations, the reliability of the examination is questionable. In connection with the participation in expert assessment of persons who do not have expert work experience in the process of forming a collective expert assessment, a wide range of expert assessments is possible.

The aim and objectives of research
The aim of research is to develop methodological support to improve the quality of collective expert assessment of business processes.
To achieve this aim, it is necessary to solve the following objectives: 1. Analyze the existing methods of collective expert assessment when making management decisions.
2. Build a methodology for collective expert assessment with increased reliability when making management decisions by an expanded team of experts.
3. Carry out approbation of the developed methodology at a really functioning enterprise.

Research of existing solutions of the problem
The most famous and popular method of collective expert assessment is the Delphi method [4]. Although this method has been in use for over 50 years, it is considered an expert technique to provide estimates with a high level of confidence [5]. The Delphi method involves a team of highly qualified experts, is interactive and is conducted in several rounds. It continues until the required level of reliability of the expert judgment is reached [6]. The reliability of the Delphi method is determined by the size of the statistical confidence interval, which fits the estimates of all experts. The last decade has seen a sharp rejection and intense criticism of the Delphi method. The disadvantages of the Delphi method are its theoretical groundlessness and large financial costs for the application of the method. «From the mid-60s to the end of the 20th century, the Delphic method was used in scientific research at least 40,000 times with an average cost of 5,000 USD per examination, with the involvement of a large number of experts, the cost of examination increased to 130,000 USD [7]. A definite alternative to the Delphi method was the methods of strict mathematical formalism describing the process of collective expert assessment. The first publication in this direction should be considered [8]. Further development of mathematical methods for processing the results of collective expertise as a matrix structure in rank scales was obtained in [9]. Until recently, the question of the reliability of the results of the collective examination remained open. However, this issue was resolved to a certain extent with the introduction of the concept of collective decision consensus [10]. In particular, in [11] it is shown that the reliability of an expert assessment is determined by the level of consensus among experts -by consensus let's mean complete consistency of the vectors of rank assessments of all experts with respect to all evaluated alternatives. In [12], 10 postulates are formulated for which a consensus is reached. It is also shown there that a strong consensus is achievable only when the Kemeny median is used to aggregate the individual rankings of experts. The Kemeny median is a ranking that is equidistant in some fixed measure from the individual rankings of experts. It is shown that the calculation of the Kemeny median is a problem of non-polynomial complexity (NP-problem that requires large computational costs [13]. However, in [14], algorithms for the accelerated calculation of the Kemeny median are given. Analysis of the literature shows that there are prerequisites for improving the quality of collective expert assessment, however, this issue has not been sufficiently investigated, both theoretically and practically, and this research became the content of this work.

Methods of research
The traditional mathematical model of the formation of a collective expert assessment of managerial decisionmaking is as follows [15]. K-experts participating in the examination are invited to evaluate and arrange, in accordance with personal preferences, N options for management decisions (or actions that should be taken to achieve the goal of the examination), usually called alternatives and forming a variety of alternatives ,..., , i N = 1, . Each of the K-experts considers the presented alternatives in the rank scale [16], that is, each alternative is associated with a natural number Rank N i = 1, , called the rank of the alternative and determining the place of each alternative in the vector of alternatives ordered in accordance with the preferences of the given expert, that is, the best alternative gets Rank 1 1 = , the next -Rank 2 2 = , the worst alternative -Rank N N = . Thus, an individual ranking of the j-th expert is formed j K = ( ) 1,..., : which is an N-element row vector. Let's note that in this paper it is assumed that experts can't repeat the same rank estimate for different alternatives, i. e., there are no related ranks. Further, let's assume that only the construction of the Kemeny median is used as the aggregation function [14].
According to the recommendations [10,12], before calculating the consensus ranking RANK CONS , it is neces sary to assess the quality or the achievable level of consensus. The quality of consensus is determined by the integral measure of the similarity of matrix rows RANK CONS . A number of different new estimates of the similarity of rows of the consensus ranking matrix have been proposed in the literature. These estimates are based on different distances between individual expert rankings [17]. Nevertheless, the most used practically as a measure of consensus is the Kendall ranked concordance coefficient, or simply the concordance coefficient [18]: where C ij -rank of the i-th alternative in the individual ranking of the j-th expert, i. e. C R ank . In [19] it is shown that in some special cases the Kendall concordance coefficient gives biased estimates, and a new coefficient of consistency of the matrix of expert assessments is proposed. However, the proposed coefficient has a high computational complexity. Therefore, in what follows, it is the Kendall coefficient in the form (3) that is applied.
The values of W lie in the range [0; 1], the closer the value of W is to one, the higher the consistency level of the matrix RANK CONS and the higher the quality of the consensus reached as a result of collective expert assessment.
The values of the concordance coefficient W ∈[ ; . ] 0 0 3 indicate a low level of consensus, i. e., a low agreement of expert opinions, and values W ∈[ . ; ] 0 7 1 about a high level of consensus or a high agreement of expert opinions [19]. Thus, the result of the examination is expressed by two quantitative estimates RANK CONS and the corresponding value of the concordance coefficient W. Fig. 1 shows a general scheme of the collective expert assessment procedure.
In contrast to the formulated procedure of collective expert assessment for an extended team of experts, the following methodology for increasing the reliability of consensus has been developed. is estimated. Since the rankings are presented in rank scales, traditional correlation coefficients are inapplicable to them, and in this case, Spearman's rank correlation coefficient is sequentially calculated:   Next, the calculated coefficient value r S is compared with the experimentally determined threshold value r S Thress = 0 9 . , and if the condition r r S j S Thress < is met, a statistical hypothesis is accepted that the ranking of the expert with the number j is in conflict with the aggregated consensus ranking. Thus, the competence level of the expert with number j is recognized as insufficient and the row with his/her ranking is removed from the matrix of expert rankings RANK EXP _ .

K N ×
As a result, a matrix of expert rankings RANK EXP _ ′× K N is formed with a reduced dimension ( ) , ′ < K K but with an increased consensus level. The concordance coefficient ′ W is recalculated, which characterizes the quality (or reliability of the consensus). In this case, always ′ > W W, that is, the reliability of the examination increases. The calculated value τ K is compared with the experimentally determined threshold value determined experimentally τ K Thresh = 0 95 . , and if the condition τ τ K j K Thresh < is met, the statistical hypothesis is again accepted that the ranking of the expert with the number j is in conflict with the aggregated consensus ranking. The competence level of an expert already with a different number j is again recognized as insufficient and the row with his/her ranking is removed from the matrix of expert rankings RANK EXP _ , ISSN 2664-9969 are calculated for it. Wherein ′′ > ′ > W W W . Thus, as a result, it is possible to obtain the maximum consensus and the most reliable result of the examination with an expanded team of experts.

Research results
The proposed method was tested at a small business enterprise «New BTI» PLC (Odesa, Ukraine). «New BTI» enterprise provides a full range of services for real estate registration, technical inventory and operations with land. The collective assessment was organized in order to determine the best strategy for overcoming the crisis situation in which the company found itself as a result of prolonged quarantine (during 2020, the company's profit fell by 23.6 %). The examination was carried out by an expanded team of experts in the amount of 20 people. Of these, 5 participants are company managers, experienced workers with extensive experience in business structures of the relevant profile. The rest of the participants are employees with different qualifications, length of service and work experience, who wished to take part in the search for the best strategy for overcoming the crisis.
The experts were asked to assess, on a five-point rank scale, 5 strategies presented by experts as alternatives (Table 1). After the examination, its results were processed according to the developed methodology described in Section 5. Fig. 2 shows graphical representations of the matrices of expert rankings RANK EXP _ , which make it possible to assess the degree of consistency of the matrix of individual expert assessments achieved at each stage. In the graphical presentation of the ranking matrices in the individual rankings of experts, the worst score is automatically removed by the data processing and presentation program. Fig. 2 clearly shows a gradual increase in the consistency of collective expert assessment. This indicates a gradual increase in the reliability of the examination. As a result of the examination, the following final ranking was obtained: The following values of the concordance coefficient were achieved at the appropriate stages of the examination: -

SWOT analysis of research results
Strengths. The strength of the research is the achieved possibility of increasing the reliability of the collective expert decision. The reliability can be increased to values of the coefficient W > 0.9 even with a large scatter of individual expert rankings.
Weaknesses. The weak side is that the formation of a collective expert assessment requires a rather complex mathematical processing of the results of the examination. This requires the use of special software.
Opportunities. A promising direction in further research is the creation of network remote systems for decision support in business processes. This will provide an opportunity for all persons wishing to take part in resolving a problem situation to take part in the examination via the Internet. The creation of such systems is of interest for small and medium-sized businesses not only in Ukraine, but also in international business projects.
Threats. The threat to the conducted research is that many persons with insufficient expert qualifications can take part in the collective assessment during the examination with an expanded composition of experts. Identifying them as experts with insufficient qualifications requires additional processing of the expertise data. This increases the processing time of the results of collective expert assessment and, accordingly, reduces its efficiency.

Conclusions
1. The existing methods of collective expert assessment of decision-making strategies in business structures are analyzed. It has been established that the methods used today are insufficiently substantiated mathematically. They are also very expensive. So, the average costs reach 5,000 USD per examination, with the involvement of a larger number of experts, the cost of the examination rises to 130,000 USD. An alternative option is to conduct an expert examination with an expanded composition of experts. In this case, not only professional experts participate in the expert assessment, but all persons wishing to participate in solving the problem. This significantly reduces the cost of the examination. However, due to the participation of persons with insufficient qualifications in the assessment, the reliability of the examination results is questionable. Methods for improving the quality of examination for this option are unknown.
2. A method of collective expert assessment with increased reliability in making managerial decisions by an expanded team of experts has been developed. It is proposed to evaluate the quality or reliability of the examination by the Kendall concordance coefficient W. The value of the coefficient W varies within [0; 1]. The developed technique consists of successive stages. At the beginning, individual expert rankings for which Spearman's rank correlation coefficient is less than the threshold value of 0.9 are excluded from the general consensus matrix. A further increase in reliability is achieved by excluding from the general matrix of expert assessments those individual expert rankings for which the value of the rank correlation coefficient -Kendall tau is less than the threshold value equal to 0.95. As a result, the developed technique makes it possible to increase the concordance coefficient to values [0.7; 1], indicating a high reliability of the examination results. 3. The developed technique was tested at the small business enterprise «New BTI» (Odesa, Ukraine). The examination was carried out by an expanded team of experts in the amount of 20 people. The experts were asked to evaluate 5 options for the strategies of the enterprise in crisis conditions, presented by the experts as alternatives. At the initial stage, the reliability of the examination, assessed by the concordance coefficient, was W = 0.36. As a result of using the developed methodology for data processing, the reliability of the examination was increased to the value W = 0.91. This value of the concordance coefficient indicates the achievement of almost complete consensus. The consensus level and the reliability of the collective expert decision were increased by 0.55. Approbation of the developed methodology at a real enterprise has shown its practical effectiveness.