COMPARATIVE ASSESSMENT OF THE COMPETENCE OF EXPERTS IN THE FIELD OF HIGHER EDUCATION BY DIFFERENT METHODS

The group expert assessment is one of the effective methods for obtaining reliable data for solving problem issues. A group of experts should be well chosen to obtain agreed conclusions. Otherwise, there may be problems with determining the quantitative measure of the consistency of expert opinions. An analysis of the quantitative degree of consistency allows a more substantiated interpretation of the reasons for the discrepancy of the findings. In this case, the competence of the experts greatly affects the results of expert assessment [1–4]. The expert level provides for the involvement of highly skilled and competent specialists who are both theoretically and practically well informed about the specific issues in a particular field of activity. Independent information on the competence of an expert in any field of activity is possible to obtain by means of a quantitative assessment. For this purpose, special questionnaires are used. Correctly formulated questionnaire for interviewing experts is important for reliable assessments [1, 4, 5]. It is equally important to choose the most appropriate method for assessing the competence of experts in higher education. An increase in the reliability of the data can be achieved through the use of specialized software. In [4], the authors conducted research and selected the most appropriate methods for evaluating the competence of experts, as well as proposed appropriate specialized software. COMPARATIVE ASSESSMENT OF THE COMPETENCE OF EXPERTS IN THE FIELD OF HIGHER EDUCATION BY DIFFERENT METHODS


Introduction
The group expert assessment is one of the effective methods for obtaining reliable data for solving problem issues.A group of experts should be well chosen to obtain agreed conclusions.Otherwise, there may be problems with determining the quantitative measure of the consistency of expert opinions.An analysis of the quantitative degree of consistency allows a more substantiated interpretation of the reasons for the discrepancy of the findings.In this case, the competence of the experts greatly affects the results of expert assessment [1][2][3][4].
The expert level provides for the involvement of highly skilled and competent specialists who are both theoretically and practically well informed about the specific issues in a particular field of activity.Independent information on the competence of an expert in any field of activity is possible to obtain by means of a quantitative assessment.For this purpose, special questionnaires are used.Correctly formulated questionnaire for interviewing experts is important for reliable assessments [1,4,5].
It is equally important to choose the most appropriate method for assessing the competence of experts in higher education.An increase in the reliability of the data can be achieved through the use of specialized software.In [4], the

Literature review and problem statement
The idea of introducing Quality Management Systems (QMS) in Higher Education Institutions (HEI) in accordance with the requirements of the international standard ISO 9001 has not been widely used at the time.In [6], this is explained by the fact that in addition to the process approach, which is fundamental to the standards of the ISO 9000 series, the human aspect is also important in solving the problem of providing the necessary quality of education.The latter is the result of the work of skilled, well-trained specialists, who should be interested not only in providing high-quality services, but also in continuous improvement of the level of qualification.
The analysis of the state of education in Ukraine, given in [7], showed the absence of a system for monitoring the quality of the national education system.More than 70 % of the population believes that higher education in Ukraine has low and medium quality.Two-thirds of employers consider the quality of training and qualifications of Ukrainian graduates to be low or inadequate to meet production needs.
The phrase "quality assurance" in the context of European standards covers such processes as quality assessment, accreditation and audit.European standards for quality assurance in higher education are developed by the European Network for Quality Assurance in Higher Education (ENQA) [8].The standards relate to: internal quality assurance in HEI, external quality assurance in higher education, quality assurance in the activities of the External Quality Assurance Agencies (EQAA).
HEIs should carefully approach the selection of teachers, checking their level of competence, and create conditions for further training [8].No less attention is paid to the specialists of EQAA.After all, the results of the Agency's review should not only ensure the validity and reliability of these processes, but also create a European dimension in ensuring the quality of higher education.Particular attention is paid to the careful selection of competent staffs of the expert level, including the international one, who must have the necessary skills and knowledge for the effective organization and implementation of external quality assurance management of HEI.
A thorough analysis of the appropriate methods for assessing the competence of experts in different fields of activity, advantages and disadvantages of each method were the subject of previous studies [4]: -method of ranking -does not provide sufficient accuracy of ranking objects, the number of which is more than 15-20 [1,2,5,9]; -method of direct evaluation -cannot be used in case of the incomplete awareness of an expert about the investigated properties of the object [1,3,9]; -method of successive comparison [5] -most labor-intensive and complex; -method of pairwise comparison [5,[10][11][12] -in comparison with other methods is quite simple, characterized by the highest level of authenticity of estimation results and allows investigating plenty of objects with great accuracy.
European quality standards reflect the exemplary experience of external quality assurance from across Europe.However, they do not provide detailed guidance as to what exactly should be checked, how to carry out an external check.This is a sphere of national autonomy.Although, the exchange of information between agencies and government must eventually lead to the appearance of similar elements.
The analysis showed that in most developed countries of the world, as well as in European countries, the issue of the formation of the EQAA personnel is specific for each country.In this case, the qualification requirements and criteria for selecting candidates are based on a qualitative approach without the necessary quantitative characteristics to be assessed.There are practically no scientific publications regarding the criteria for evaluating specialists in the field of higher education.National laws and regulations state that the selection of the relevant personnel is the prerogative of a specific HEI.
In [13], the selection procedures by the Brazilian Ministry of Education for experts-auditors of HEI by indexes and the specific weights of variables for: experience of management, teaching, position, number of publications, etc. are considered.Evaluations are carried out on a scale from 1 to 5, where 3 is the minimum satisfactory level.According to the proposed procedure, 25 % of the weight to choose the respondent is given by the variables such as the experience of managing the course and the availability of the position of the president or member of the course, and 30 % -the experience associated with institutional management.However, setting such weight for variables may lead to incorrect selection of specialists, since it gives more priority to managerial skills than to other competencies.
In [14,15], the competence approach in the context of providing higher education quality assurance on the use of qualitative assessments is considered, but there are no quantitative assessments of the competence of experts.
In the framework of the Bologna Process on the establishment of the European Higher Education Area, the National Agency for Quality Assurance of Higher Education (NAQAHE) was established in 2014 in Ukraine.The qualification requirements for candidates, who on a competitive basis are selected as members of the NAQAHE, are formed.There are also general criteria for selecting candidates according to the appropriate ball scales.However, the appropriate scale for evaluation by each of the criteria is not set.
Today, there are practically no methods and quantitative criteria for assessing the competence of specialists in the field of higher education, which determines the relevance of the research.Therefore, it is currently important to select the appropriate methods for assessing the competence of the experts of the NAQAHE and the lecturers of HEI.
Quantitative characteristics for the competence of experts can be estimated using universal statistical software.However, the application of specially developed software can significantly improve the efficiency of expert assessment methods and eliminate errors in calculating the results.To implement the process of assessing the competence of experts, universal statistical analysis programs, including IBM SPSS Statistics 22 (USA) and Microsoft Excel 2010 (USA) can be used.As a result, quantitative results of such assessment can be obtained using specially designed criteria for competence assessment.However, this may limit the resulting score to just a simple arithmetic mean or the frequency of all criteria for each expert without the appropriate ranking of the results of competence assessment.In addition, such an assessment requires considerable time expenditures.
For research of complex objects or systems, the analytic hierarchy process (АНР) was chosen.This method is a mathematical instrument of systems approach to the complex problems of decision-making.AHP allows finding interactively such a version (alternative), which is best consistent with its understanding of the problem, and with the requirements for its solution.It is worth noting that the methods of analysis of the root cause, scenario, influence on activity, causal relationships do not provide for numerical estimates [3,19,20].The basic AHP [16][17][18] and its modifications [21] are applicable only in the case of a small number of given alternatives and do not make it possible to combine different opinions of expert groups.Therefore, on the basis of the AHP described in [22][23][24][25], specialized software "Competence AHP 1.0" (Ukraine) was developed, which can be used to assess the competence of experts in different fields of activity.
In [1,19,20], attention is drawn to the number of experts involved in the survey, which significantly affects the accuracy of group assessment.The number of experts for the formation of an expert group in practice can range from 10 to 150 people.A decrease in the number of experts involved may decrease the accuracy of the assessment, and an increase, although increases the accuracy of the evaluation, but complicates the organization of the questionnaire.Therefore, when choosing the number of experts, it is necessary to settle for a compromise between the accuracy and complexity of the work.
Specialized software for evaluating the competence of experts can significantly simplify the process of evaluation, increase reliability and have a number of other advantages compared with universal statistical software.Application of the software allows more reasonable selection of the most competent experts to form a group for evaluating certain problem issues in a particular field of activity.
In [22][23][24][25], the methodology for quantifying the competence of experts, taking into account the characteristics of data uncertainty (DU), which can be used for a comparative assessment of the competence level of experts in various fields of activity, is described.Appropriate criteria for the ball's assessment of the competence of experts in a particular field of activity to implement this methodology are established.Specialized software "Competence DU 2.1" (Ukraine) on the basis of the method described in [22][23][24][25] was developed.In the methodology and software "Competence DU 2.1", a number of mathematical indicators for each expert, the reference value of the estimation and its general standard uncertainty are calculated.The software provides a quantitative assessment of the competence of experts by establishing the required level of competence.
It is expedient to conduct a study on the suitability of existing methods for assessing the competence of specialists in the field of higher education.In doing so, it is necessary to identify and apply the relevant quantitative (ball) characteristics (criteria) for the competence of the expert.This will allow not only the selection of highly qualified specialists in the field of higher education, but also will provide other benefits.There is an opportunity to convincingly justify the superiority of one candidate for expert over the other based on objective data, rather than on the basis of sympathy or subjective opinions.

The aim and objectives of the study
The aim of the work is to determine the most appropriate method for assessing the competence of experts in the field of higher education using science-based criteria.
For the achievement of the aim, the following objectives were accomplished: -to assess the competence of experts in higher education through the selected methods; -to conduct a comparative analysis of the results obtained with the use of the selected methods of assessing the competence of experts; -to assess the convergence of the results and suitability of the selected methods for assessing the competence of experts in higher education.

Materials and methods of research for the development of methods for assessing the competence of experts in the field of higher education
The selection of specific specialists for the assessment of problem issues is carried out on the basis of analysis of certain characteristics of each of them.In addition to wellknown characteristics, experts' assessment should also be carried out according to specific criteria, in particular on higher education.To do this, it is necessary to determine which criteria should be set for specialists in the field of higher education.The scale of relative values (scores) from 1 to 9 was chosen for assessing the competence of experts (ACE), which allows evaluating both traditional methods and AHP.
The main criteria for selecting specialists given the specificity of the study were the following requirements: knowledge of higher education and availability of pedagogical experience.In order to determine the level of competence of specialists in the field of higher education, it is first necessary to determine the factors that have a significant impact on this level.These influential factors were: education; overall experience; scientific and pedagogical experience; academic degree, academic rank; position; expert experience.The established factors became the basis for the development of a special questionnaire and the formation of criteria and their rating points [26].
After performing the relevant calculations [1,19,20] and analysis of the possibility of conducting evaluation and data processing, the optimal number of the expert group -from 10 to 30 persons was determined.
Ideally, the available expert data is the average score of all the criteria of the ACE.In other cases, such data may be an exhaustive list of the sum of points for all criteria of the ACE.In any case, there will be some uncertainty about expert data.The uncertainty range in these data may be limited by using independent methods or consistency checks.A detailed description of the methods of DU and AHP was presented in [22][23][24][25].
The values of the matrix of pairwise comparisons (MPC) of the established criteria A with the normalized eigenvectors K i -priority vectors for the selected criteria (Table 1) and weight coefficients for the selected evaluation criteria W i (Table 2) are determined.

Table 1
Matrix of pairwise comparisons of the criteria Table 2 Weight coefficients for the selected evaluation criteria For comparison of results obtained by the methods of DU and АНР, recalculation of the obtained global priorities in coefficients of competence (CC) for experts (k AHP ) is performed for the method of АНР: where G ni and G max are global priorities for experts and maximum global priority for experts, respectively [4].
In the comparative analysis of results obtained by the methods of DU and АНР, variation (dispersion) of the CC for the experts was calculated where k max is the maximal CC (equals 1.00); k min is the minimum CC, obtained for a certain expert.
In the future, the indicators that characterize the consistency of the data obtained with respect to the applied criteria, in particular, the Kendall consistency coefficient W, taking into account the related ranks are calculated: where where S is the sum of the squares of deviations from the mean; М is the total number of evaluated experts; N is the number of ACE criteria; T i is the total number of identical ranks for the i-th expert on all the criteria; t q is the number for the same rank for the i-th expert on all the criteria; Q is the number of groups of the same ranks in the i-th expert on the criteria; q is the same rank for the i-th expert on all criteria.The obtained value of the Kendall consistency coefficient W is analyzed and the conclusion is made on the degree of data consistency in accordance with the Margolin's scale [27].If necessary, the adjustment of the points for certain criteria is performed.
The verification of the consistency of the data obtained for each expert using the Chi-squared ( 2χ ) test is performed at a confidence level of 0.95 and according to the methodology [4]: The obtained value based on the χ 2 agreement criterion for the confidence level of 0.05 is compared with the critical value χ χ  T( , ; ) M for this confidence level.If the value of the criterion is less than the tabulated critical value, consideration of the correction of point values for certain criteria of ACE is required.
The methods of DU and АНР are appropriate to apply for a comparative ACE on the basis of objective data according to the established criteria for ACE for a specific field of activity.These methods allow the selection of the most competent experts and rejecting (if necessary) experts whose data do not meet a certain level of established requirements.

Assessment of the competence of experts in higher education by different methods
The competence of 25 experts in the field of higher education from two HEIs of Ukraine was evaluated.The evaluation was carried out according to the same set criteria: K 1 -education; K 2 -overall experience; K 3 -scientific experience; K 4 -scientific and pedagogical experience; K 5academic degree, academic rank; K 6 -position; K 7 -expert experience in the field of education.Specific points for each criterion are given in [26].
The averages of the criteria for higher education in the Microsoft Excel 2010 (USA) software were evaluated.Radar charts for the averages of the ACE criteria are shown in Fig. 1.The blue line represents the average of the criteria.
The results obtained show a small dispersion of the average values for the ACE criteria (from 3.7 to 6.6), indicating the presence of balance.
The MPC values of the A criteria with the normalized eigenvectors K i (Table 1) for the implementation of the AHP method and the weight coefficients for the selected criteria of ACE (Table 2) are determined.
For the proposed criteria of ACE, the largest eigenvalue of MPC of A criteria was λ max =8.07.Verification of consistency of output data that was used for the construction of the A matrix on the obtained consistency index I c =0.18 and consistency ratio C d =0.1 showed that the consistency ratio satisfied the requirements of consistency (C d ≤0.1).This shows the consistency of the set criteria of ACE.

Example of assessment of the competence of experts in higher education
The competence of 25 experts in higher education was evaluated using the developed questionnaire.The survey involved specialists from Ukraine: 16 from the Odesa State Academy of Technical Regulation and Quality (OSATRQ) and 9 from the Odesa National Economic University (ONEU) [26].
Using specialized software "Competence DU 2.1" and "Competence AHP 1.0" (Ukraine), quantitative indicators of the competence of experts were evaluated [22][23][24][25].The results of the assessment of the competence of these experts are shown in Table 3.
The final results of the evaluation by the software "Competence DU 2.1" are given in [22,26], and by the software "Competence AHP 1.0" -in Fig. 2.
The column diagram in Fig. 2 displays the ranking of experts in order of decreasing values of the CC.On the x-axis, the experts are represented, on the y-axis -CC of experts (in points).When constructing the chart, the Pareto principle or the "20/80" principle is used, which generally means that 20 % of the effort gives 80 % of the result (yellow columns), and the remaining 80 % of the effortonly 20 % of the result (blue columns) [28].Experts who meet the requirements of the Pareto principle may, in the first place, be recommended for participation in expert groups of a certain level.
In Fig. 2, blue columns mark the experts who, according to the Pareto principle, have high competence, and yellow columns -lower competence.That is, experts 03, 09, 01, 06, 07, 04, 08, 20, 05, 17 and 11 have a high level of competence, and experts 12, 21, 10, 24, 14, 15 and 13 -a lower competence.The latter may be rejected or, if necessary, will require measures to improve the level of competence.
In order to compare the results obtained using the methods of DU and AHP, the AHP method reassesses the global priorities received for experts in the CC (k AHP ) using the expression (1).Comparison of the results obtained with the help of the specialized software "Competence DU 2.1" and "Competence AHP 1.0" (Ukraine) are shown in Fig. 3.
The data on the CC of experts, calculated using the specialized software "Competence DU 2.1", are marked with a blue line, and the red dotted line is the data, calculated using the specialized software "Competence AHP 1.0".The correlation of the results of the assessment of the CC of experts by the DU and AHP methods is evident (Fig. 3).
The dispersion (variation) of the CC for the HEI experts is calculated with the use of (2).The obtained values of the dispersion of CC and the minimum CC for HEI, obtained by the methods of DU and AHP, are shown in Table 4.
The average value of the dispersion of CC for all experts of HEI is obtained: by the DU method -0.68; by the AHP method -0.76.The average value of the minimum CC for all experts of HEI is obtained: by the DU method -0.33; by the AHP method -0.24.Consistency of the obtained results for HEI with the use of the Kendall consistency coefficient W (k=n-1=6, α=0.01) by (3) was evaluated and verification of data consistency by the χ 2 agreement criterion by (4) was performed.The values of the Kendall consistency coefficients W and the χ 2 agreement criterion for the HEI experts are shown in Table 5.
The obtained Kendall consistency coefficient W=0.55 and agreement criterion χ 2 =91.72 for all experts correspond to the set requirements for the confidence level of 0.05 (χ χ

Discussion of the results of assessment of the competence of experts in the field of higher education
From the total number of evaluated experts, experts (highlighted in gray) number 13, 14, 15, 24, 10, 21 and 12 (28 %) were rejected by all software, as can be seen from Table 3.Moreover, "Competence AHP 1.0" software rejected expert's number 16, 18, 23 and 25 (additionally 16 %).
It can be noted that 7 experts were rejected (by at least one software) on the basis of the obtained results.The percentage of rejected experts based on the use of specific software ranges from 28 % (7 experts from 25) to 44 % (11 experts from 25).As a whole, it can be concluded that the results of the evaluation are highly consistent: 7 out of 25 (28 %) for "Competence DU 2.1" and "Competence AHP 1.0" software.
Experts were also asked to evaluate their own competence during this questionnaire.Based on the results obtained, they overestimated their competence in comparison with the obtained objective estimates of 19 experts from 25 (76 %), the difference is from 0.08 to 0.50 relative points.An interesting fact is that the latter applies to all 11 rejected experts on the basis of objective evaluation (the difference is from 0.24 to 0.5 relative points) [26].A general comparison of the CC of experts in higher education obtained using the methods of DU and AHP shows a clear correlation of the obtained values with the experts (Table 4).At the same time, for the AHP method, there is a larger dispersion of CC values (R AHP ) than for the DU (R DU ) method.Also, for the AHP method, the minimum CC (k AHP ) is smaller than for the DU (k DU ) method.This can be explained by the greater number of rejected experts for the AHP method (44 %) than for the DU method (28 %).
Dispersion of the CC values of experts in the field of higher education for all HEI, obtained using: -DU method: maximum -0.71; minimum -0.64; -AHP method: maximum -0.88; minimum -0/64.Minimum CC of experts in the field of higher education for all HEI, obtained using: -DU method: maximum -0.36; minimum -0.29; -AHP method: maximum -0.36; minimum -0.12.
The obtained results (Table 2) show a large correlation of the applied methods of DU and AHP.It is advisable to apply the methods considered for the comparative assessment of the competence of experts in the field of higher education on the basis of objective data and according to the specially set criteria of the ACE.It should be noted that the method of AHP to a lesser extent allows for the consideration of less competent experts than the DU method.This is evidenced by the lower value of the CC for this method than for the DU method.
The considered methods allow selecting the most competent experts and rejecting specialists whose data do not meet the established level of requirements.In this case, the quantitative data obtained objectively will allow justifying the superiority of one specialist over the other, excluding the possibility of influence of subjective thoughts or sympathies.This will allow for the formation of groups for the assessment of specific problem issues in the field of higher education, obtaining more reliable expert assessments or ensuring a more skilled work of agencies, specialized commissions or councils.
In the future, on the basis of the conducted studies, it is expedient to consider the criteria in more detail in order to identify more and less significant.It is also advisable to consider the impact, of both the removal of certain criteria, and the introduction of additional criteria in solving certain problems on research results.

Conclusions
1.The most effective methods of expert assessment that are suitable for assessing the competence of experts in the field of higher education are considered.The methods of DU and AHP are chosen as the most suitable.Using the selected methods, assessment of the competence of experts in higher education according to the established criteria was carried out.The results were processed using specialized software "Competence DU 2.1" and "Competence AHP 1.0" (Ukraine).The estimated average ratio for the criteria used for higher education ACEs has shown a small dispersion of averages for the ACE criteria (from 3.7 to 6.6), indicating a good balance.
2. A comparative analysis of the results showed the convergence and correlation of the obtained values with the experts.Of the total number of evaluated experts, 7 experts (28 %) were rejected by all software used.In addition, 4 experts (16 % more) were rejected by the "Competence AHP 1.0" software.However, the AHP method, to a lesser extent, allows for the consideration of less competent experts than the DU method.This is evidenced by a lower CC for this method than for the DU method.Therefore, it is advisable to use the DU and AHP methods for comparative assessment of the competence of experts based on objective data according to the established criteria of higher education qualifications.The DU method allows the selection of the most competent experts and rejecting experts whose data do not meet a certain level of established requirements.The verification of the consistency of the data obtained using the Kendall consistency coefficients W showed the average level of consistency of the data obtained with experts, and according to the χ 2 agreement criterion -the full compliance with the criterion.
3. In general, one can state the high coincidence of the results of quantitative evaluation of the level of competence of experts, conducted with the help of specialized software, based on different techniques of ACE.This indicates the suitability of the applied methods for ACE in the field of higher education.The application of the DU and AHP methods makes it possible to more reasonably select the most competent experts to form a group for evaluating specific problem issues in the field of higher education.As a result, this will lead to more reliable group expert assessments, which will provide more qualified work of agencies, specialized commissions or councils.

Fig. 1 .
Fig. 1.Average value of the criteria for assessing the competence of experts indicates the high consistency of data for experts.

Fig. 2 .Fig. 3 .
Fig. 2. View of the window of the software "Competence AHP 1.0" with the final results

Table 3
Results of assessment of the competence of experts in the field of higher education by different software

Table 5
Kendall consistency coefficient W and χ 2 agreement criterion for the experts of higher educational institutions