A comparative analysis of the assessment results of the competence of technical experts by methods of analytic hierarchy process and with using the Rasch model

Authors

  • Oleh Velychko State Enterprise ‘‘All-Ukrainian State Scientific and Production Centre for Standardization, Metrology, Certification and Protection of Consumer’’, (SE “Ukrmetrteststandard”) Metrolohychna str., 4, Kyiv, Ukraine, 03143, Ukraine https://orcid.org/0000-0002-6564-4144
  • Tetyana Gordiyenko Odessa State Academy of Technical Regulation and Quality Kovalska str., 15, Odessa, Ukraine, 65020, Ukraine https://orcid.org/0000-0003-0324-9672

DOI:

https://doi.org/10.15587/1729-4061.2018.131459

Keywords:

analytical hierarchy, Rasch model, characteristic curve, competence of experts, scale of evaluation, software

Abstract

Known scales (criteria) for assessing the competence of experts in the field of technical regulation using the method of analytical hierarchy process (AHP) and Rasch model are investigated. The main features of constructing the mathematical Rash model are presented. The results of the analysis of scales for assessing the competence of experts in the field of technical regulation on a specific example of questioned specialists in measurement of time and frequency were considered. The results were processed using the specialized software “Competence MAI 1.1” (Ukraine) and MINISTEP 4.0.1 (USA).

A comparative analysis of the results was carried out in order to determine the effectiveness of the assessment scales. The obtained results showed the possibility of applying the Rasch model for the analysis of the scale of expert assessment in the field of technical regulation. The analysis of the results obtained on the multidimensional Rasch model showed that the chosen criteria scale for experts corresponds to the requirements set by the Rash model. The obtained measurement data for this model allow you to calculate the established statistics for both the criteria and for the evaluated experts.

A comparative analysis of the results obtained with the use of AHP and Rasch model showed convergence, suitability and correlation of the obtained values for experts. Only two out of twenty one (9.5 %) evaluated experts have data that are unsuitable for the analysis by the Rasch model, which indicates a low level of competence. AHP to a lesser extent allows for the consideration of less competent experts than with the use of the Rash model. This is evidenced by a lower coefficient of competence for the AHP than in the application of the Rasch model. AHP and Rasch model should be used in various fields of activity as a useful tool for comparative assessment of the competence of technical experts on the basis of objective data according to established criteria.

Author Biographies

Oleh Velychko, State Enterprise ‘‘All-Ukrainian State Scientific and Production Centre for Standardization, Metrology, Certification and Protection of Consumer’’, (SE “Ukrmetrteststandard”) Metrolohychna str., 4, Kyiv, Ukraine, 03143

Doctor of Technical Sciences, Professor, Director

Scientific and Production Institute of Electromagnetic Measurements

Tetyana Gordiyenko, Odessa State Academy of Technical Regulation and Quality Kovalska str., 15, Odessa, Ukraine, 65020

Doctor of Technical Sciences, Associate Professor, Head of Department

Department of standardization, conformity assessment and quality

References

  1. Velychko, O. M., Gordiyenko, T. B., Kolomiets, L. V. (2015). Methodologies of expert’s competence evaluation and group expert evaluation. Metallurgical and Mining Industry, 2, 262–271.
  2. Velychko, O., Gordiyenko, T. (2015). Evaluation of competence of the experts in field of metrology and instrumentations. XXI IMEKO World Congress “Measurement in research and industry”. Prague, Czech Republic, 5.
  3. Velychko, O. N., Gordienko, T. B., Karpenko, S. R., Kolomiets, L. V. (2016). Evaluation of experts competence on the measurement of electrical power using the method of analytic hierarchy. Metallurgical and Mining Industry, 11, 70–76.
  4. Velychko, O., Gordiyenko, T., Kolomiets, L. (2017). A comparative analysis of the assessment results of the competence of technical experts by different methods. Eastern-European Journal of Enterprise Technologies, 4 (3 (88)), 4–10. doi: 10.15587/1729-4061.2017.106825
  5. Velychko, O., Gordiyenko, T., Kolomiets, L. (2017). A comparative analysis of results of the group expert assessment of metrological assurance of measurements. Eastern-European Journal of Enterprise Technologies, 6 (9 (90)), 30–37. doi: 10.15587/1729-4061.2017.114468
  6. Saaty, T. L. (1992). The Hierarchon: A Dictionary of Hierarchies. Pittsburgh, Pennsylvania: RWS Publications, 510.
  7. Saati, T. L. (2008). Prinyatie resheniy pri zavisimostyah i obratnyh svyazyah: Analiticheskie seti. Moscow: Izd-vo LKI, 360.
  8. Drake, P. R. (1998). Using the Analytic Hierarchy Process in Engineering Education. International Journal of Engineering Education, 14 (3), 191–196.
  9. Chernysheva, T. Yu. (2009). Ierarhicheskaya model' ocenki i otbora ekspertov. Doklady TUSUR. Upravleniya, vychislitel'naya tekhnika i informatika, 1 (19), 168–173.
  10. Kolpakova, T. A. (2011). Opredelenie kompetentnosti ekspertov pri prinyatii gruppovyh resheniy. Radioelektronika, informatyka, upravlinnia, 1, 40–43.
  11. Kalinina, I. O., Hozhyi, O. P., Musenko, H. O. (2013). Vrakhuvannia kompetentnosti ekspertiv u metodakh bahatokryterialnoho analizu v zadachakh ratsionalnoho vyboru. Nauk. pratsi Chornomor. derzh. univer. Kompiuterni tekhnolohiyi, 191 (179), 116–123.
  12. Bhushan, N. (2004). Strategic Decision Making: Applying the Ana­lytic Hierarchy Process. London: Springer-Verlag, 170. doi: 10.1007/b97668
  13. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institution for Educational Research, 199.
  14. Andrich, D. (1988). Rasch models for measurement. Newbury Park CA: Sage, 95. doi: 10.4135/9781412985598
  15. Bond, T. G., Fox, C. M. (2007). Applying the Rasch model: fundamental measurement in the human sciences. 2nd ed. Psychology Press, 360. doi: 10.4324/9781410614575
  16. Hagquist, C., Bruce, M., Gustavsson, J. P. (2009). Using the Rasch model in nursing research: An introduction and illustrative example. International Journal of Nursing Studies, 46 (3), 380–393. doi: 10.1016/j.ijnurstu.2008.10.007
  17. Ehlan, A. H., Kucukdeveci, A. A., Tennant, A. (2010). The Rasch Measurement Model. Research Issues in Physical & Rehabilitation Medicine. Pavia: Maugeri Foundation, 89–102.
  18. Demenchenok, O. G. (2010). Matematicheskie osnovy Rasch Measurement. Pedagogicheskie izmereniya, 1.
  19. Wright, B. D., Linacre, J. M. (1987). A measurement is the quantification of a specifically defined comparison. Rasch model derived from objectivity. Rasch Measurement Transactions, 1 (1), 4–5.
  20. A User’s Guide to WINSTEPS®MINISTEP Rasch-Model Computer Programs. Program Manual 4.0.0 by John M. Linacre (2017).
  21. Rasch Model/Rasch Analysis: Definition, Examples. Statistics How To. Available at: http://www.statisticshowto.com/rasch-model
  22. Communication Validity and Rating Scales. Institute for Objective Measurement. Available at: http://www.rasch.org/rmt/rmt101k.htm
  23. Velychko, O., Gordiyenko, T. (2016). The evaluation of activity of Technical Committees of Standardization for Metrology and Measurement on national level. Journal of Physics: Conference Series, 772, 012007. doi: 10.1088/1742-6596/772/1/012007

Downloads

Published

2018-05-17

How to Cite

Velychko, O., & Gordiyenko, T. (2018). A comparative analysis of the assessment results of the competence of technical experts by methods of analytic hierarchy process and with using the Rasch model. Eastern-European Journal of Enterprise Technologies, 3(3 (93), 14–21. https://doi.org/10.15587/1729-4061.2018.131459

Issue

Vol. 3 No. 3 (93) (2018): Control processes

Section

Control processes

License

Copyright (c) 2018 Oleh Velychko, Tetyana Gordiyenko

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.

A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.