Development of the method of distances for processing expert estimates in information systems

Authors

DOI:

https://doi.org/10.15587/1729-4061.2019.163964

Keywords:

expert estimates, estimation of indicators, on-line assessment, processing of expert estimate, analysis of expert estimation, information technology

Abstract

Current research focuses on expert estimation of indicators by employing a limited number of qualified experts, which involves considerable time and cost. The advent of modern information technology allows rapid and efficient assessment of indicators that characterize performance of enterprises. The main tool for solving the issue related to the credibility of such an estimation is the development of new methods for processing its results.

We have proposed a method of distances to treat the results from expert estimations of indicators and examined the results of its work.

To this end, theoretical substantiation of the method has been performed, based on the concept of proximity (distances) among estimates relative to the average value, coefficients of experts' competence, and normalization of the point scale of assessment.

We have explored three variants of expert point- and verbal-based estimates of independent and dependent indicators in line with the method of distances, which showed that the convergence rate of iteration process for all cases varies from 1 to 4 orders of magnitude at each iteration. That makes it possible to draw a conclusion about a very small number of computations by the information system and sufficient speed of processing expert estimates.

A comparative analysis of the method of distances with a similar method of square deviations has revealed almost the same rate of convergence up to 3‒5 iterative steps, but the proposed method yields an estimate close to the average estimate of each indicator. It also enables the processing of results in information systems by 99 % faster, from dozens, even hundreds of times, more estimates than that in the methods of expert selection, which is important under current competitive environment. Average efficiency of the method compared to the method of expert selection is 5.8 %

Author Biographies

Tetyana Otradska, Odessa College of Computer Technologies “Server” Polskyi uzviz str., 1, Odessa, Ukraine, 65026

PhD, Director

Natalia Shybaieva, Odessa National Polytechnic University Shevchenka ave., 1, Odessa, Ukraine, 65044

PhD, Associate Professor

Department of Information Technologies

Oleksandr Shyrshkov, Odessa College of Computer Technologies “Server” Polskyi uzviz str., 1, Odessa, Ukraine, 65026

PhD, Associate Professor, Head of Department

Department of Information Technologies

Mykola Rudnichenko, Odessa National Polytechnic University Shevchenka ave., 1, Odessa, Ukraine, 65044

PhD, Associate Professor

Department of Information Technologies

Ashot Halustian, Odessa College of Computer Technologies “Server” Polskyi uzviz str., 1, Odessa, Ukraine, 65026

Lecturer

Department of Information Technologies

References

  1. Blikhar, T. (2013). Monitorynh navchalnykh dosiahnen u vyshchykh navchalnykh zakladakh: pidkhody, zdobutky. Monitorynh yakosti vyshchoi osvity: dosvid roboty, innovatsiyi, problemy: materialy I Vseukrainskoi naukovo-praktychnoi internet-konferentsiyi. Irpin: NU DPSU, 129–130.
  2. Odainyk, S. (2010). Monitorynh yakosti pedahohichnoi osvity: teoretychnyi aspekt. Molod i rynok, 9, 64–67.
  3. Zakharova, A. A., Grigorjeva, A. A., Tseplit, A. P., Ozgogov, E. V. (2016). Models Used to Select Strategic Planning Experts for High Technology Productions. IOP Conference Series: Materials Science and Engineering, 127, 012029. doi: https://doi.org/10.1088/1757-899x/127/1/012029
  4. Liljelind, I. E., Rappaport, S., Levin, J., Pettersson Strömbäck, A. E., Sunesson, A.-L. K., Järvholm, B. G. (2001). Comparison of self-assessment and expert assessment of occupational exposure to chemicals. Scandinavian Journal of Work, Environment & Health, 27 (5), 311–317. doi: https://doi.org/10.5271/sjweh.619
  5. Wieck, M. M., McLaughlin, C., Chang, T. P., Rake, A., Park, C., Lane, C. et. al. (2018). Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment. The American Journal of Surgery, 216 (3), 630–635. doi: https://doi.org/10.1016/j.amjsurg.2018.01.010
  6. Zakharova, A. A., Ostanin, V. V. (2015). Formalization model of expert knowledge about a technical index level of engineering products. IOP Conference Series: Materials Science and Engineering, 91, 012070. doi: https://doi.org/10.1088/1757-899x/91/1/012070
  7. Dugarova, D. T., Starostina, S. E., Cherepanova, L. V. (2016). Expert Training as a Factor of Independent Assessment of Quality of Social Services. Scholarly Notes of Transbaikal State University. Series Philosophy. Cultural Studies. Sociology. Social Work, 11 (2), 170–179.
  8. Cooke, R. M., Goossens, L. L. H. J. (2008). TU Delft expert judgment data base. Reliability Engineering & System Safety, 93 (5), 657–674. doi: https://doi.org/10.1016/j.ress.2007.03.005
  9. Mach, K. J., Mastrandrea, M. D., Freeman, P. T., Field, C. B. (2017). Unleashing expert judgment in assessment. Global Environmental Change, 44, 1–14. doi: https://doi.org/10.1016/j.gloenvcha.2017.02.005
  10. Kozierkiewicz-Hetmańska, A. (2017). The analysis of expert opinions’ consensus quality. Information Fusion, 34, 80–86. doi: https://doi.org/10.1016/j.inffus.2016.06.005
  11. Christoforaki, M., Ipeirotis, P. G. (2015). A system for scalable and reliable technical-skill testing in online labor markets. Computer Networks, 90, 110–120. doi: https://doi.org/10.1016/j.comnet.2015.05.020
  12. Cain, M., McLeay, S. (2016). Statistical Auditing of Non-transparent Expert Assessments. Sankhya B, 78 (2), 362–385. doi: https://doi.org/10.1007/s13571-016-0124-8
  13. Ageev, A. S. (2011). Technique of carrying out of expert estimations of activity of aviation enterprise on safety of flights. Nauchniy vestnik Moskovskogo gosudarstvennogo tekhnicheskogo universiteta grazhdanskoy aviacii, 174, 69–72.
  14. Kolpakova, T. A. (2011). Determination of experts competence in group decision-making. Radioelektronika, informatika, upravlenie, 1, 40–43.
  15. Roberts, T. S. (2004). A ham sandwich is better than nothing: Some thoughts about transitivity. Australian Senior Mathematics Journal, 18 (2), 60–64.
  16. Podd'yakov, A. N. (2006). Intransitive character of superiority relations and decision-making. Psihologiya. Zhurnal Vysshey shkoly ekonomiki, 3 (3), 88–111.
  17. Otradskaya, T., Gogunskii, V., Antoshchuk, S., Kolesnikov, O. (2016). Development of parametric model of prediction and evaluation of the quality level of educational institutions. Eastern-European Journal of Enterprise Technologies, 5 (3 (83)), 12–21. doi: https://doi.org/10.15587/1729-4061.2016.80790
  18. Gül, E., Çokluk, Ö., Gül, Ç. D. (2015). Development of an Attitudes Scale toward Online Assessment. Procedia – Social and Behavioral Sciences, 174, 529–536. doi: https://doi.org/10.1016/j.sbspro.2015.01.699
  19. Formanek, M., Wenger, M. C., Buxner, S. R., Impey, C. D., Sonam, T. (2017). Insights about large-scale online peer assessment from an analysis of an astronomy MOOC. Computers & Education, 113, 243–262. doi: https://doi.org/10.1016/j.compedu.2017.05.019

Downloads

Published

2019-04-12

How to Cite

Otradska, T., Shybaieva, N., Shyrshkov, O., Rudnichenko, M., & Halustian, A. (2019). Development of the method of distances for processing expert estimates in information systems. Eastern-European Journal of Enterprise Technologies, 2(3 (98), 40–47. https://doi.org/10.15587/1729-4061.2019.163964

Issue

Vol. 2 No. 3 (98) (2019): Control processes

Section

Control processes

License

Copyright (c) 2019 Tetyana Otradska, Natalia Shybaieva, Oleksandr Shyrshkov, Mykola Rudnichenko, Ashot Halustian

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.

A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.