Development of a complex solution for a human-robot interaction and operators training using VR-integrated DT framework

Authors

DOI:

https://doi.org/10.15587/2706-5448.2026.352356

Keywords:

digital twin, user interaction, immersive learning, virtual reality, augmented reality

Abstract

The object of research is the process of managing human-robot interaction through the virtual reality-integrated digital twin system. The problem addressed in the research is that, despite active development of architectural solutions, there are barriers to fundamental research and operator training due to the prohibitive costs, technical complexity, and proprietary restrictions of industrial-grade robotic hardware.

The obtained results are the creation of a comprehensive, scalable, and flexible digital twin architecture, implemented through a functional prototype. The prototype digital twin framework is an extensible tool for testing research strategies and can be adapted for specific tasks or equipment. The implementation synchronizes a low cost 6-degrees-of-freedom manipulator with its digital model using the Unreal Engine. Analysis of the application areas of the developed system highlights the potential of virtual reality to improve human-robot interaction.

These results were made possible by a complex approach combining architectural design and experimental prototyping. Unlike industrial solutions, which focus on specific technologies, a general approach to system design was applied. A significant advantage of focusing on general principles is that they can be developed without recourse to using complex real systems, which are associated with safety, accessibility, and cost issues.

The proposed solution is designed to enable systematic testing of a wide range of user interface designs, situational awareness tools, interaction, and collaboration strategies in a risk-free virtual environment. The underlying design and software will be publicly available, enabling researchers to use a standardized yet flexible approach to develop human-robot interaction systems based on the results presented in this research.

Author Biographies

Yevhen Palazhchenko, Sumy State University

PhD Student

Department of Information Technologies

Vira Shendryk, Sumy State University

PhD, Associate Professor, Professor of Department

Department of Information Technologies

 

Georg Meyer, University of Liverpool

PhD in Technical Science, Associate Professor, Honorary Research Professor

Faculty of Engineering

References

  1. Grieves, M., Vickers, J. (2016). Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems. Transdisciplinary Perspectives on Complex Systems. Springer International Publishing, 85–113. https://doi.org/10.1007/978-3-319-38756-7_4
  2. Tao, F., Zhang, H., Liu, A., Nee, A. Y. C. (2019). Digital Twin in Industry: State-of-the-Art. IEEE Transactions on Industrial Informatics, 15 (4), 2405–2415. https://doi.org/10.1109/tii.2018.2873186
  3. Bruynseels, K., Santoni de Sio, F., Van den Hoven, J. (2018). Digital Twins in Health Care: Ethical Implications of an Emerging Engineering Paradigm. Frontiers in Genetics, 9. https://doi.org/10.3389/fgene.2018.00031
  4. Jeddoub, I., Nys, G.-A., Hajji, R., Billen, R. (2023). Digital Twins for cities: Analyzing the gap between concepts and current implementations with a specific focus on data integration. International Journal of Applied Earth Observation and Geoinformation, 122, 103440. https://doi.org/10.1016/j.jag.2023.103440
  5. Lamagna, M., Groppi, D., Nezhad, M. M., Piras, G. (2021). A comprehensive review on digital twins for smart energy management system. International Journal of Energy Production and Management, 6 (4), 323–334. https://doi.org/10.2495/eq-v6-n4-323-334
  6. Meyer, G. F., Wong, L. T., Timson, E., Perfect, P., White, M. D. (2012). Objective Fidelity Evaluation in Multisensory Virtual Environments: Auditory Cue Fidelity in Flight Simulation. PLoS ONE, 7 (9), e44381. https://doi.org/10.1371/journal.pone.0044381
  7. Cooper, N., Millela, F., Cant, I., White, M. D., Meyer, G. (2021). Transfer of training – Virtual reality training with augmented multisensory cues improves user experience during training and task performance in the real world. PLOS ONE, 16 (3), e0248225. https://doi.org/10.1371/journal.pone.0248225
  8. Cooper, N., Milella, F., Pinto, C., Cant, I., White, M., Meyer, G. (2018). The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment. PLOS ONE, 13 (2), e0191846. https://doi.org/10.1371/journal.pone.0191846
  9. Longo, F., Nicoletti, L., Padovano, A. (2017). Smart operators in industry 4.0: A human-centered approach to enhance operators’ capabilities and competencies within the new smart factory context. Computers & Industrial Engineering, 113, 144–159. https://doi.org/10.1016/j.cie.2017.09.016
  10. Palazhchenko, Y., Shendryk, V., Shendryk, S. (2023). Digital Twins Data Visualization Methods. Problems of Human Interaction: A Review. New Technologies, Development and Application VI. Springer Nature Switzerland, 478–485. https://doi.org/10.1007/978-3-031-31066-9_53
  11. Ordaz, N., Romero, D., Gorecky, D., Siller, H. R. (2015). Serious Games and Virtual Simulator for Automotive Manufacturing Education & Training. Procedia Computer Science, 75, 267–274. https://doi.org/10.1016/j.procs.2015.12.247
  12. Checa, D., Bustillo, A. (2019). A review of immersive virtual reality serious games to enhance learning and training. Multimedia Tools and Applications, 79 (9-10), 5501–5527. https://doi.org/10.1007/s11042-019-08348-9
  13. Bucchiarone, A. (2022). Gamification and virtual reality for digital twin learning and training: architecture and challenges. Virtual Reality & Intelligent Hardware, 4 (6), 471–486. https://doi.org/10.1016/j.vrih.2022.08.001
  14. Stark, R., Damerau, T. (2019). Digital Twin. CIRP Encyclopedia of Production Engineering. Berlin, Heidelberg: Springer, 1–8. https://doi.org/10.1007/978-3-642-35950-7_16870-1
  15. Palazhchenko, Y., Shendryk, V., Ivanov, V., Hatala, M. (2023). Industry 5.0: Aspects of Collaboration Technologies. Flexible Automation and Intelligent Manufacturing: Establishing Bridges for More Sustainable Manufacturing Systems. Springer Nature Switzerland, 609–617. https://doi.org/10.1007/978-3-031-38165-2_71
  16. Wright, L., Davidson, S. (2020). How to tell the difference between a model and a digital twin. Advanced Modeling and Simulation in Engineering Sciences, 7 (1). https://doi.org/10.1186/s40323-020-00147-4
  17. Jones, D., Snider, C., Nassehi, A., Yon, J., Hicks, B. (2020). Characterising the Digital Twin: A systematic literature review. CIRP Journal of Manufacturing Science and Technology, 29, 36–52. https://doi.org/10.1016/j.cirpj.2020.02.002
  18. Tao, F., Xiao, B., Qi, Q., Cheng, J., Ji, P. (2022). Digital twin modeling. Journal of Manufacturing Systems, 64, 372–389. https://doi.org/10.1016/j.jmsy.2022.06.015
  19. Garg, G., Kuts, V., Anbarjafari, G. (2021). Digital Twin for FANUC Robots: Industrial Robot Programming and Simulation Using Virtual Reality. Sustainability, 13 (18), 10336. https://doi.org/10.3390/su131810336
  20. Wang, Z., OuYang, Y., Kochan, O. (2023). Bidirectional Linkage Robot Digital Twin System Based on ROS. 2023 17th International Conference on the Experience of Designing and Application of CAD Systems (CADSM), 1–5. https://doi.org/10.1109/cadsm58174.2023.10076497
  21. Inamura, T., Mizuchi, Y. (2021). SIGVerse: A Cloud-Based VR Platform for Research on Multimodal Human-Robot Interaction. Frontiers in Robotics and AI, 8. https://doi.org/10.3389/frobt.2021.549360
  22. Acker, J., Rogers, I., Guerra-Zubiaga, D., Tanveer, M. H., Moghadam, A. A. A. (2023). Low-Cost Digital Twin Approach and Tools to Support Industry and Academia: A Case Study Connecting High-Schools with High Degree Education. Machines, 11 (9), 860. https://doi.org/10.3390/machines11090860
  23. Vairavasamy, S., J, N. I., MJ, H., Ahmed, S., N, S., MM Dean, R. (2022). Simulation And Real Time Of VR Controlled Robotic Manipulator Using ROS. 2022 IEEE Bombay Section Signature Conference (IBSSC). IEEE, 1–6. https://doi.org/10.1109/ibssc56953.2022.10037520
  24. Palazhchenko, Y. (2026). Digital-twin-system-for_learning-strategies-development-in-VR. Available at: https://github.com/palazhchenko/Digital-twin-system-for_learning-strategies-development-in-VR
  25. Sailer, M., Homner, L. (2019). The Gamification of Learning: a Meta-analysis. Educational Psychology Review, 32 (1), 77–112. https://doi.org/10.1007/s10648-019-09498-w
Development of a complex solution for a human-robot interaction and operators training using VR-integrated DT framework

Downloads

Published

2026-02-28

How to Cite

Palazhchenko, Y., Shendryk, V., & Meyer, G. (2026). Development of a complex solution for a human-robot interaction and operators training using VR-integrated DT framework. Technology Audit and Production Reserves, 1(2(87), 11–17. https://doi.org/10.15587/2706-5448.2026.352356

Issue

Vol. 1 No. 2(87) (2026): Information and control systems

Section

Information Technologies

License

Copyright (c) 2026 Yevhen Palazhchenko, Vira Shendryk, Georg Meyer

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.