A comparative analysis of the assessment results of the competence of technical experts by different methods

Authors

  • Oleh Velychko State Enterprise ‘‘All-Ukrainian State Scientific and Production Centre for Standardization, Metrology, Certification and Protection of Consumer’’ (SE “Ukrmetrteststandard”) Metrolohychna str., 4, Kyiv, Ukraine, 03143, Ukraine https://orcid.org/0000-0002-6564-4144
  • Tetyana Gordiyenko Odessa State Academy of Technical Regulation and Quality Kovalska str., 15, Odessa, Ukraine, 65020, Ukraine https://orcid.org/0000-0003-0324-9672
  • Leonid Kolomiets Odessa State Academy of Technical Regulation and Quality Kovalska str., 15, Odessa, Ukraine, 65020, Ukraine https://orcid.org/0000-0003-2341-3345

DOI:

https://doi.org/10.15587/1729-4061.2017.106825

Keywords:

competence, expert, evaluation criteria, technical regulation, data uncertainties, analytic hierarchy

Abstract

Present methods and facilities for the evaluation of expert’s competence are researched. The most suitable methods for the evaluation of expert’s competence – the methods of data uncertainties and Analytic Hierarchy Process are analysed in detail and selected. Evaluation of expert’s competence in the field of technical regulation (time and frequency measurement) according to the set criteria was carried out. The results were processed by special methodologies, and also universal and specialized software. The specialized software facilities take into account the data uncertainties and application of the method of analytic hierarchy process. With the application of the universal software Microsoft Excel (USA), the ratio of the averages for the criteria used for the evaluation of expert’s competence in the field of technical regulation was estimated.

Comparative analyses of the results obtained using the methods for the evaluation of expert’s competence taking into account the data uncertainties and applying the method of analytic hierarchy process was carried out. The results of the analysis showed the convergence and correlation of the data obtained, and also confirmed the fitness of the methods for the evaluation of expert’s competence. The results showed a small dispersion of the average values for the criteria of evaluation of expert’s competence in the limits from 5.2 to 7.9, which testifies to a good balance. Dispersion of the values obtained using the given methods shows correlation.

The methods for the evaluation of expert’s competence which take into account the data uncertainties and application of the method of analytic hierarchy process,are expedient to apply as a useful tool for the comparative estimation of expert’s competence on the basis of objective data according to the set criteria for different fields (spheres) of activity. This will assist the increase of evaluation authenticity and will allow forming more competent groups of experts.

Author Biographies

Oleh Velychko, State Enterprise ‘‘All-Ukrainian State Scientific and Production Centre for Standardization, Metrology, Certification and Protection of Consumer’’ (SE “Ukrmetrteststandard”) Metrolohychna str., 4, Kyiv, Ukraine, 03143

Doctor of Technical Sciences, Professor, Director

Scientific and Production Institute of Electromagnetic Measurements

Tetyana Gordiyenko, Odessa State Academy of Technical Regulation and Quality Kovalska str., 15, Odessa, Ukraine, 65020

Doctor of Technical Sciences, Associate Professor, Head of Department

Department of standardization, conformity assessment and quality

Leonid Kolomiets, Odessa State Academy of Technical Regulation and Quality Kovalska str., 15, Odessa, Ukraine, 65020

Doctor of Technical Sciences, Professor, Rector

References

  1. Velychko O. M., Gordiyenko T. B., Kolomiets L. V. (2015). Methodologies of expert’s competence evaluation and group expert evaluation. Metallurgical and Mining Industry, 2, 262–271.
  2. Litvak, B. G. (1996). Ekspertnye otsenki i prinyatiye resheniy. Мoscow: Patent, 271.
  3. Larichev, O. I. (2002). Teoriya i metody prinyatiya resheniy. Moscow: Logos, 392.
  4. Pavlov, A. N., Sokolov, B. V. (2005). Metody obrabotki ehkspertnoy informacii. Sankt-Peterburg: GUAP, 42.
  5. Chernysheva, T. Yu. (2009). Ierarhicheskaya model' ocenki i otbora ehkspertov. Doklady TUSUR. Upravleniya, vychislitel'naya tekhnika i informatika, 1 (19), 168–173.
  6. Orlov, A. I. (2002). Ehkspertnye ocenki. Moscow, 31.
  7. Kolpakova, T. A. (2011). Opredelenie kompetentnosti ehkspertov pri prinyatii gruppovyh resheniy. Radioelektronika, informatyka, upravlinnia, 1, 40–43.
  8. Xia, M., Chen, J. (2015). Consistency and consensus improving methods for pairwise comparison matrices based on Abelian linearly ordered group. Fuzzy Sets and Systems, 266, 1–32. doi: 10.1016/j.fss.2014.07.019
  9. Koczkodaj, W. W., Szybowski, J., Wajch, E. (2016). Inconsistency indicator maps on groups for pairwise comparisons. International Journal of Approximate Reasoning, 69, 81–90. doi: 10.1016/j.ijar.2015.11.007
  10. Malovik, K. N. (2013). Razvitie nauchnyh osnov povysheniya kachestva ocenivaniya i prognozirovaniya resursnyh harakteristik slozhnyh ob’ektov. Sevastopol': SNUYAEHiP, 332.
  11. Hrabovetskyi, B. Ye. (2010). Metody ekspertnykh otsinok: teoriia, metodolohiia, napriamy vykorystannia. Vinnytsia: VNTU, 171.
  12. Kalinina, I. O., Hozhyi, O. P., Musenko, H. O. (2013). Vrakhuvannia kompetentnosti ekspertiv u metodakh bahatokryterialnoho analizu v zadachakh ratsionalnoho vyboru. Nauk. pratsi Chornomor. derzh. univer. Kompiuterni tekhnolohii, 191 (179), 116–123.
  13. Saaty, T. L. (1992). The Hierarchon: A Dictionary of Hierarchies. Pittsburgh, Pennsylvania: RWS Publications, 496.
  14. Saaty, T. L. (2008). Prinyatie resheniy pri zavisimostyah i obratnyh svyazyah: Analiticheskie seti. Moscow: Izd-vo LKI, 360.
  15. Drake, P. R. (1998). Using the Analytic Hierarchy Process in Engineering Education. International Journal of Engineering Education, 14 (3), 191–196.
  16. Roy, R. (Ed.) (2004). Strategic Decision Making: Applying the Analytic Hierarchy Process. London: Springer-Verlag, 172. doi: 10.1007/b97668
  17. Velychko, O. M., Karpenko, S. R. (2016). The group expert evaluation of the metrological assurance of electric power measurements. Journal of Physics: Conference Series, 772, 012043. doi: 10.1088/1742-6596/772/1/012043
  18. Velychko, O., Gordiyenko, T. (2015). Evaluation of competence of the experts in field of metrology and instrumentations. XXI IMEKO World Congress “Measurement in research and industry”. Prague, Czech Republic.
  19. Podinovskiy, V. V., Nogin, V. D. (1982). Pareto-optimal'nye resheniya mnogokriterial'nyh zadach. Moscow: Nauka, 256.
  20. Velychko, O. N., Gordienko, T. B., Karpenko, S. R., Kolomiets, L. V. (2016). Evaluation of experts competence on the measurement of electrical power using the method of analytic hierarchy. Metallurgical and Mining Industry, 11, 70–76.
  21. Velychko, O., Gordiyenko, T. (2016). The evaluation of activity of Technical Committees of Standardization for Metrology and Measurement on national level. Journal of Physics: Conference Series, 772, 012007. doi: 10.1088/1742-6596/772/1/012007

Downloads

Published

2017-08-31

How to Cite

Velychko, O., Gordiyenko, T., & Kolomiets, L. (2017). A comparative analysis of the assessment results of the competence of technical experts by different methods. Eastern-European Journal of Enterprise Technologies, 4(3 (88), 4–10. https://doi.org/10.15587/1729-4061.2017.106825

Issue

Vol. 4 No. 3 (88) (2017): Control processes

Section

Control processes

License

Copyright (c) 2017 Oleh Velychko, Tetyana Gordiyenko, Leonid Kolomiets

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.

A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.