Development of methodological support for improving the quality of expert assessment of business processes

Authors

DOI:

https://doi.org/10.15587/2706-5448.2021.225336

Keywords:

collective expert assessment, expanded team of experts, a variety of alternatives, concordance coefficient, examination reliability

Abstract

The object of research is the process of forming a collective expert assessment with increased reliability in making management decisions in business structures by an expanded team of experts. One of the most problematic places in the expert assessment of management decisions is the complexity of forming a competent expert team and the rather high cost of the expertise. In recent years, there has been a tendency for expert assessment with an expanded team of experts. In this case, not only professional experts are involved in the examination, but also all persons wishing to take part in solving the problem. In this case, the reliability of the examination raises doubts. In connection with the participation in expert assessment of persons who do not have experience in expert work, a wide range of expert assessments is possible. The analysis of the current state of the methods of expert assessment in business is carried out. It has been established that the Delphi method, which was most used until recently, does not meet modern requirements. More progressive methods are based on mathematical consensus theory. Consensus is understood as the degree of correlation of individual expert assessments performed in rank scales. In the course of the study, formalized mathematical approaches to the organization of collective expertise were used. A method for processing the results of an examination with an expanded composition of experts was developed. The developed methodology is focused on identifying experts with insufficient qualifications. The methodology allows for a step-by-step assessment of the reliability of the collective expert decision by assessing the Kendall concordance coefficient. It is shown that the phased exclusion of assessments by experts with insufficient qualifications allows increasing the level of consensus, the quality and reliability of the collective expert assessment. The developed methodology has been tested in a really functioning enterprise to make a decision on the exit strategy of the enterprise from their crisis. The use of the developed methodology has made it possible to significantly increase the reliability of the examination results, assessed by the concordance coefficient. The results are useful for practical application in business structures when conducting expert examinations involving a wide range of participants.

Author Biographies

Vitalii Antoshchuk, Odessa National Polytechnic University

Assistant

Department of International Management and Innovation

Volodymyr Filippov, Odessa National Polytechnic University

Doctor of Economics Sciences, Associate Professor

Department of International Management and Innovation

Varvara Kuvaieva, Odessa National Polytechnic University

PhD, Senior Lecturer

Department of Information Systems

References

  1. Budescu, D. V., Chen, E. (2015). Identifying Expertise to Extract the Wisdom of Crowds. Management Science, 61 (2), 267–280. doi: http://doi.org/10.1287/mnsc.2014.1909
  2. Dong, Y., Zha, Q., Zhang, H., Kou, G., Fujita, H., Chiclana, F., Herrera-Viedma, E. (2018). Consensus reaching in social network group decision making: Research paradigms and challenges. Knowledge-Based Systems, 162, 3–13. doi: http://doi.org/10.1016/j.knosys.2018.06.036
  3. Gubanov, D., Korgin, N., Novikov, D., Raikov, A. (2014). E-Expertise: Modern Collective Intelligence. Studies in Computational Intelligence. Cham: Springer. doi: http://doi.org/10.1007/978-3-319-06770-4
  4. Rowe, G., Wright, G.; Armstrong, J. S. (Ed.) (2001). Expert Opinions in Forecasting: The Role of the Delphi Technique. Principles of Forecasting. International Series in Operations Research & Management Science, 30. Boston: Springer, 125–144. doi: http://doi.org/10.1007/978-0-306-47630-3_7
  5. Skinner, R., Nelson, R. R., Chin, W. W., Land, L. (2015). The Delphi Method Research Strategy in Studies of Information Systems. Communications of the Association for Information Systems, 37, 31–63. doi: http://doi.org/10.17705/1cais.03702
  6. Kauko, K., Palmroos, P. (2014). The Delphi method in forecasting financial markets – An experimental study. International Journal of Forecasting, 30 (2), 313–327. doi: http://doi.org/10.1016/j.ijforecast.2013.09.007
  7. Kuznetsov, A. B., Boiarinov, G. A., Mukhin, A. S., Simutis, I. S. (2017). Informativity of a single collective expert assessment in predicting treatment outcomes of a patient in critical condition. Modern problems of science and education, 6. Available at: http://www.science-education.ru/ru/article/view?id=27047
  8. Cook, W. D., Kress, M., Seiford, L. M. (1997). A general framework for distance-based consensus in ordinal ranking models. European Journal of Operational Research, 96 (2), 392–397. doi: http://doi.org/10.1016/0377-2217(95)00322-3
  9. Bury, H., Wagner, D. (2003). Application of Kemeny’s Median for Group Decision Support. Applied Decision Support with Soft Computing, 124, 235–262. doi: http://doi.org/10.1007/978-3-540-37008-6
  10. Nguyen, N. T. (2008). Advanced Methods for Inconsistent Knowledge Management. London: Springer-Verlag, 351. doi: http://doi.org/10.1007/978-1-84628-889-0
  11. Dong, Y., Xu, J. (2016). Consensus Building in Group Decision Making. Singapore: Springer, 201. doi: http://doi.org/10.1007/978-981-287-892-2
  12. Dang, D. T., Nguyen, N. T., Hwang, D. (2019). Increasing the Quality of Multi-step Consensus. Lecture Notes in Computer Science. Yogyakarta, 3–14. doi: http://doi.org/10.1007/978-3-030-14802-7_1
  13. Boltenkov, V. A., Kuvaeva, V. I., Pozniak, A. V. (2017). Analiz mediannykh metodov konsensusnogo agregirovaniia rangovykh predpochtenii. Іnformatika ta matematichnі metodi v modeliuvannі, 7 (4), 307–317.
  14. Boltenkov, V., Kuvaieva, V., Galchonkov, O., Ishchenko, A. (2018). The research of possibilities for fast calculation of median consensus rankings. Eastern-European Journal of Enterprise Technologies, 4 (4 (94)), 27–35. doi: http://doi.org/10.15587/1729-4061.2018.140686
  15. Triantaphyllou, E., Hou, F., Yanase, J. (2020). Analysis of the Final Ranking Decisions Made by Experts After a Consensus has Been Reached in Group Decision Making. Group Decision and Negotiation, 29 (2), 271–291. doi: http://doi.org/10.1007/s10726-020-09655-5
  16. Kuznetsov, M. P., Strijov, V. V. (2014). Methods of expert estimations concordance for integral quality estimation. Expert Systems with Applications, 41 (4), 1988–1996. doi: http://doi.org/10.1016/j.eswa.2013.08.095
  17. Del Moral, M. J., Chiclana, F., Tapia, J. M., Herrera-Viedma, E. (2018). A comparative study on consensus measures in group decision making. International Journal of Intelligent Systems, 33 (8), 1624–1638. doi: http://doi.org/10.1002/int.21954
  18. Abdi, H.; Salkind, N. (Ed.) (2007). The Kendall Rank Correlation Coefficient. Encyclopedia of Measurement and Statistics. SAGE Publications, Inc., 1–19. doi: http://doi.org/10.4135/9781412952644.n239
  19. Venediktov, A. A. (2018). O pokazatele soglasovannosti ekspertnykh otsenok. Vooruzhenie i ekonomika, 3 (45), 52–66.

Downloads

Published

2021-02-26

How to Cite

Antoshchuk, V., Filippov, V., & Kuvaieva, V. (2021). Development of methodological support for improving the quality of expert assessment of business processes. Technology Audit and Production Reserves, 1(4(57), 22–27. https://doi.org/10.15587/2706-5448.2021.225336

Issue

Vol. 1 No. 4(57) (2021): Economics of enterprises. Macroeconomics

Section

Economic Cybernetics: Original Research

License

Copyright (c) 2021 Віталій Миколайович Антощук, Володимир Юрійович Філіппов, Варвара Ігорівна Куваєва

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.