Development of the method of distances for processing expert estimates in information systems

Authors

DOI:

https://doi.org/10.15587/1729-4061.2019.163964

Keywords:

expert estimates, estimation of indicators, on-line assessment, processing of expert estimate, analysis of expert estimation, information technology

Abstract

Current research focuses on expert estimation of indicators by employing a limited number of qualified experts, which involves considerable time and cost. The advent of modern information technology allows rapid and efficient assessment of indicators that characterize performance of enterprises. The main tool for solving the issue related to the credibility of such an estimation is the development of new methods for processing its results.

We have proposed a method of distances to treat the results from expert estimations of indicators and examined the results of its work.

To this end, theoretical substantiation of the method has been performed, based on the concept of proximity (distances) among estimates relative to the average value, coefficients of experts' competence, and normalization of the point scale of assessment.

We have explored three variants of expert point- and verbal-based estimates of independent and dependent indicators in line with the method of distances, which showed that the convergence rate of iteration process for all cases varies from 1 to 4 orders of magnitude at each iteration. That makes it possible to draw a conclusion about a very small number of computations by the information system and sufficient speed of processing expert estimates.

A comparative analysis of the method of distances with a similar method of square deviations has revealed almost the same rate of convergence up to 3‒5 iterative steps, but the proposed method yields an estimate close to the average estimate of each indicator. It also enables the processing of results in information systems by 99 % faster, from dozens, even hundreds of times, more estimates than that in the methods of expert selection, which is important under current competitive environment. Average efficiency of the method compared to the method of expert selection is 5.8 %

Author Biographies

Tetyana Otradska, Odessa College of Computer Technologies “Server” Polskyi uzviz str., 1, Odessa, Ukraine, 65026

PhD, Director

Natalia Shybaieva, Odessa National Polytechnic University Shevchenka ave., 1, Odessa, Ukraine, 65044

PhD, Associate Professor

Department of Information Technologies

Oleksandr Shyrshkov, Odessa College of Computer Technologies “Server” Polskyi uzviz str., 1, Odessa, Ukraine, 65026

PhD, Associate Professor, Head of Department

Department of Information Technologies

Mykola Rudnichenko, Odessa National Polytechnic University Shevchenka ave., 1, Odessa, Ukraine, 65044

PhD, Associate Professor

Department of Information Technologies

Ashot Halustian, Odessa College of Computer Technologies “Server” Polskyi uzviz str., 1, Odessa, Ukraine, 65026

Lecturer

Department of Information Technologies

References

  1. Blikhar, T. (2013). Monitorynh navchalnykh dosiahnen u vyshchykh navchalnykh zakladakh: pidkhody, zdobutky. Monitorynh yakosti vyshchoi osvity: dosvid roboty, innovatsiyi, problemy: materialy I Vseukrainskoi naukovo-praktychnoi internet-konferentsiyi. Irpin: NU DPSU, 129–130.
  2. Odainyk, S. (2010). Monitorynh yakosti pedahohichnoi osvity: teoretychnyi aspekt. Molod i rynok, 9, 64–67.
  3. Zakharova, A. A., Grigorjeva, A. A., Tseplit, A. P., Ozgogov, E. V. (2016). Models Used to Select Strategic Planning Experts for High Technology Productions. IOP Conference Series: Materials Science and Engineering, 127, 012029. doi: https://doi.org/10.1088/1757-899x/127/1/012029
  4. Liljelind, I. E., Rappaport, S., Levin, J., Pettersson Strömbäck, A. E., Sunesson, A.-L. K., Järvholm, B. G. (2001). Comparison of self-assessment and expert assessment of occupational exposure to chemicals. Scandinavian Journal of Work, Environment & Health, 27 (5), 311–317. doi: https://doi.org/10.5271/sjweh.619
  5. Wieck, M. M., McLaughlin, C., Chang, T. P., Rake, A., Park, C., Lane, C. et. al. (2018). Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment. The American Journal of Surgery, 216 (3), 630–635. doi: https://doi.org/10.1016/j.amjsurg.2018.01.010
  6. Zakharova, A. A., Ostanin, V. V. (2015). Formalization model of expert knowledge about a technical index level of engineering products. IOP Conference Series: Materials Science and Engineering, 91, 012070. doi: https://doi.org/10.1088/1757-899x/91/1/012070
  7. Dugarova, D. T., Starostina, S. E., Cherepanova, L. V. (2016). Expert Training as a Factor of Independent Assessment of Quality of Social Services. Scholarly Notes of Transbaikal State University. Series Philosophy. Cultural Studies. Sociology. Social Work, 11 (2), 170–179.
  8. Cooke, R. M., Goossens, L. L. H. J. (2008). TU Delft expert judgment data base. Reliability Engineering & System Safety, 93 (5), 657–674. doi: https://doi.org/10.1016/j.ress.2007.03.005
  9. Mach, K. J., Mastrandrea, M. D., Freeman, P. T., Field, C. B. (2017). Unleashing expert judgment in assessment. Global Environmental Change, 44, 1–14. doi: https://doi.org/10.1016/j.gloenvcha.2017.02.005
  10. Kozierkiewicz-Hetmańska, A. (2017). The analysis of expert opinions’ consensus quality. Information Fusion, 34, 80–86. doi: https://doi.org/10.1016/j.inffus.2016.06.005
  11. Christoforaki, M., Ipeirotis, P. G. (2015). A system for scalable and reliable technical-skill testing in online labor markets. Computer Networks, 90, 110–120. doi: https://doi.org/10.1016/j.comnet.2015.05.020
  12. Cain, M., McLeay, S. (2016). Statistical Auditing of Non-transparent Expert Assessments. Sankhya B, 78 (2), 362–385. doi: https://doi.org/10.1007/s13571-016-0124-8
  13. Ageev, A. S. (2011). Technique of carrying out of expert estimations of activity of aviation enterprise on safety of flights. Nauchniy vestnik Moskovskogo gosudarstvennogo tekhnicheskogo universiteta grazhdanskoy aviacii, 174, 69–72.
  14. Kolpakova, T. A. (2011). Determination of experts competence in group decision-making. Radioelektronika, informatika, upravlenie, 1, 40–43.
  15. Roberts, T. S. (2004). A ham sandwich is better than nothing: Some thoughts about transitivity. Australian Senior Mathematics Journal, 18 (2), 60–64.
  16. Podd'yakov, A. N. (2006). Intransitive character of superiority relations and decision-making. Psihologiya. Zhurnal Vysshey shkoly ekonomiki, 3 (3), 88–111.
  17. Otradskaya, T., Gogunskii, V., Antoshchuk, S., Kolesnikov, O. (2016). Development of parametric model of prediction and evaluation of the quality level of educational institutions. Eastern-European Journal of Enterprise Technologies, 5 (3 (83)), 12–21. doi: https://doi.org/10.15587/1729-4061.2016.80790
  18. Gül, E., Çokluk, Ö., Gül, Ç. D. (2015). Development of an Attitudes Scale toward Online Assessment. Procedia – Social and Behavioral Sciences, 174, 529–536. doi: https://doi.org/10.1016/j.sbspro.2015.01.699
  19. Formanek, M., Wenger, M. C., Buxner, S. R., Impey, C. D., Sonam, T. (2017). Insights about large-scale online peer assessment from an analysis of an astronomy MOOC. Computers & Education, 113, 243–262. doi: https://doi.org/10.1016/j.compedu.2017.05.019

Downloads

Published

2019-04-12

How to Cite

Otradska, T., Shybaieva, N., Shyrshkov, O., Rudnichenko, M., & Halustian, A. (2019). Development of the method of distances for processing expert estimates in information systems. Eastern-European Journal of Enterprise Technologies, 2(3 (98), 40–47. https://doi.org/10.15587/1729-4061.2019.163964

Issue

Section

Control processes