Research on mobile machine learning platforms for human gesture recognition in human-machine interaction systems

Authors

DOI:

https://doi.org/10.15587/2706-5448.2025.325423

Keywords:

human-machine interaction, HMI, Create ML, Google Cloud AI Platform, image processing, contactless control, ML platforms

Abstract

The subject of this research is mobile machine learning platforms for human gesture recognition within human-machine interaction systems, specifically for managing smart home components.

One of the key challenges in gesture recognition is ensuring high accuracy, efficiency, and robustness of algorithms under real-world operating conditions. The problem lies in selecting optimal machine learning platforms capable of balancing local and cloud computing, processing speed, and adaptability to changing environmental conditions.

The study presents a comparative analysis of the ML platforms Create ML (Apple) and Google Cloud AI Platform, which are used for gesture detection and recognition in smart home control systems. The obtained results demonstrate that Create ML achieves an accuracy of 95.81 %, while Google Cloud AI Platform reaches 89.43%, justifying their selection for further research. Additionally, experimental testing of sensor placement topology revealed that diagonal camera positioning increases accuracy by 0.62 % compared to parallel placement.

The increased efficiency of Create ML is due to its ability to process data locally, reducing latency and dependence on an internet connection. In contrast, Google Cloud AI Platform relies on cloud resources, enabling the processing of large volumes of data but making it dependent on data transmission speed.

The proposed gesture control algorithms can be used to enhance the accessibility of technology for people with disabilities, particularly in rehabilitation centers. Additionally, the research findings can be applied to contactless interfaces in medical facilities and public spaces, reducing the need for physical interaction with surfaces and improving hygiene levels. The use of mobile ML platforms in such scenarios allows for the optimization of computational resources and ensures the effective integration of gesture control into modern human-machine systems.

Author Biographies

Olesia Barkovska, Kharkiv National University of Radio Electronics

PhD, Associate Professor

Department of Electronic Computers

Igor Ruban, Kharkiv National University of Radio Electronics

Doctor of Technical Sciences, First Vice-Rector

 

Daria Tymoshenko, Kharkiv National University of Radio Electronics

Assistant

Department of Electronic Computers

Oleksandr Holovchenko, Kharkiv National University of Radio Electronics

Department of Electronic Computers

Oleksandr Yankovskyi , Kharkiv National University of Radio Electronics

PhD, Associate Professor

Department of Electronic Computers

References

  1. Pangarkar, T. (2025). Smart Home Statistics 2025 By Technology, Features, Security. Scoop Market. Available at: https://scoop.market.us/smart-home-statistics/ Last accessed: 07.02.2025
  2. Taiwo, O., Ezugwu, A. E., Oyelade, O. N., Almutairi, M. S. (2022). Enhanced Intelligent Smart Home Control and Security System Based on Deep Learning Model. Wireless Communications and Mobile Computing, 2022, 1–22. https://doi.org/10.1155/2022/9307961
  3. Netinant, P., Utsanok, T., Rukhiran, M., Klongdee, S. (2024). Development and Assessment of Internet of Things-Driven Smart Home Security and Automation with Voice Commands. IoT, 5 (1), 79–99. https://doi.org/10.3390/iot5010005
  4. Prieto González, L., Fensel, A., Gómez Berbís, J. M., Popa, A., de Amescua Seco, A. (2021). A Survey on Energy Efficiency in Smart Homes and Smart Grids. Energies, 14 (21), 7273. https://doi.org/10.3390/en14217273
  5. Touqeer, H., Zaman, S., Amin, R., Hussain, M., Al-Turjman, F., Bilal, M. (2021). Smart home security: challenges, issues and solutions at different IoT layers. The Journal of Supercomputing, 77 (12), 14053–14089. https://doi.org/10.1007/s11227-021-03825-1
  6. Malik, S., Shafqat, W., Lee, K.-T., Kim, D.-H. (2021). A Feature Selection-Based Predictive-Learning Framework for Optimal Actuator Control in Smart Homes. Actuators, 10 (4), 84. https://doi.org/10.3390/act10040084
  7. Aksak, N. H., Barkovska, O. Yu., Ivashchenko, H. S. (2017). Development of the hand gesture recognition system on the basis of clonal selection model. Systemy Obrobky Informatsii, 3 (149), 76–80. https://doi.org/10.30748/soi.2017.149.15
  8. Barkovska, O., Serdechnyi, V. (2024). Intelligent assistance system for people with visual impairments. Innovative technologies and scientific solutions for industries, 2 (28), 6–16. https://doi.org/10.30837/2522-9818.2024.28.006
  9. Barkovska, O., Oliinyk, D., Sorokin, A., Zabroda, I., Sedlaček, P. (2024). A System for monitoring the progress of rehabilitation of patients with musculoskeletal disorder. Advanced Information Systems, 8 (3), 13–24. https://doi.org/10.20998/2522-9052.2024.3.02
  10. Ahmed, S., Kallu, K. D., Ahmed, S., Cho, S. H. (2021). Hand Gestures Recognition Using Radar Sensors for Human-Computer-Interaction: A Review. Remote Sensing, 13 (3), 527. https://doi.org/10.3390/rs13030527
  11. Bhushan, S., Alshehri, M., Keshta, I., Chakraverti, A. K., Rajpurohit, J., Abugabah, A. (2022). An Experimental Analysis of Various Machine Learning Algorithms for Hand Gesture Recognition. Electronics, 11 (6), 968. https://doi.org/10.3390/electronics11060968
  12. Indriani, Harris, Moh., Agoes, A. S. (2021). Applying Hand Gesture Recognition for User Guide Application Using MediaPipe. Proceedings of the 2nd International Seminar of Science and Applied Technology (ISSAT 2021). https://doi.org/10.2991/aer.k.211106.017
  13. Mujahid, A., Awan, M. J., Yasin, A., Mohammed, M. A., Damaševičius, R., Maskeliūnas, R., Abdulkareem, K. H. (2021). Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model. Applied Sciences, 11 (9), 4164. https://doi.org/10.3390/app11094164
  14. Sahoo, J. P., Prakash, A. J., Pławiak, P., Samantray, S. (2022). Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network. Sensors, 22 (3), 706. https://doi.org/10.3390/s22030706
  15. Jiang, S., Kang, P., Song, X., Lo, B., Shull, P. (2022). Emerging Wearable Interfaces and Algorithms for Hand Gesture Recognition: A Survey. IEEE Reviews in Biomedical Engineering, 15, 85–102. https://doi.org/10.1109/rbme.2021.3078190
  16. Guo, L., Lu, Z., Yao, L. (2021). Human-Machine Interaction Sensing Technology Based on Hand Gesture Recognition: A Review. IEEE Transactions on Human-Machine Systems, 51 (4), 300–309. https://doi.org/10.1109/thms.2021.3086003
  17. Liu, H., Zhu, X., Lei, Z., Cao, D., Li, S. Z. (2021). Fast Adapting Without Forgetting for Face Recognition. IEEE Transactions on Circuits and Systems for Video Technology, 31 (8), 3093–3104. https://doi.org/10.1109/tcsvt.2020.3035890
  18. Uddin, Md. S., Slany, W., Hasan, K. J. (2024). On-Device Neural Network for Object Train and Recognition using Mobile. International Journal of Interactive Mobile Technologies (IJIM), 18 (12), 99–111. https://doi.org/10.3991/ijim.v18i12.47895
  19. Van der Vlist, F., Helmond, A., Ferrari, F. (2024). Big AI: Cloud infrastructure dependence and the industrialisation of artificial intelligence. Big Data & Society, 11 (1). https://doi.org/10.1177/20539517241232630
  20. Patel, D., Raut, G., Cheetirala, S. N., Nadkarni, G. N., Freeman, R., Glicksberg, B. S. et al. (2024). Cloud Platforms for Developing Generative AI Solutions: A Scoping Review of Tools and Services. arXiv preprint arXiv:2412.06044. https://doi.org/10.48550/arXiv.2412.06044
  21. Chornenkyi, V. Ya., Kazymyra, I. Ya. (2023). Research of the models for sign gesture recognition using 3D convolutional neural networks and visual transformers. Ukrainian Journal of Information Technology, 5 (2), 33–40. https://doi.org/10.23939/ujit2023.02.033
  22. Zhang, G., Ma, N., Li, J., Jiang, B., Wu, Z. (2021). Gesture Recognition Algorithm Based on Lightweight 3DCNN Network. 2021 17th International Conference on Computational Intelligence and Security (CIS), 217–221. https://doi.org/10.1109/cis54983.2021.00053
  23. Osman Hashi, A., Zaiton Mohd Hashim, S., Bte Asamah, A. (2024). A Systematic Review of Hand Gesture Recognition: An Update From 2018 to 2024. IEEE Access, 12, 143599–143626. https://doi.org/10.1109/access.2024.3421992
  24. Openja, M., Nikanjam, A., Yahmed, A. H., Khomh, F., Jiang, Z. M. J. (2022). An Empirical Study of Challenges in Converting Deep Learning Models. 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME), 13–23. https://doi.org/10.1109/icsme55016.2022.00010
  25. Wakchaure, A., Kanawade, P., Jawale, M. A., William, P., Pawar, A. B. (2022). Face Mask Detection in Realtime Environment using Machine Learning based Google Cloud. 2022 International Conference on Applied Artificial Intelligence and Computing (ICAAIC), 557–561. https://doi.org/10.1109/icaaic53929.2022.9793201
  26. Abdul-Al, M., Kyeremeh, G., Qahwaji, R., Abd-Alhameed, R. (2022). The Impacts of Depth Camera and Multi-Sensors for Face and Fingerprint Recognition in 2D and 3D – A Case Study. Proceedings of 2nd International Multi-Disciplinary Conference Theme: Integrated Sciences and Technologies, IMDC-IST 2021. Sakarya. https://doi.org/10.4108/eai.7-9-2021.2314977
  27. Matilainen, M., Sangi, P., Holappa, J., Silven, O. (2016). OUHANDS database for hand detection and pose recognition. 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA). Oulu, 1–5. https://doi.org/10.1109/ipta.2016.7821025
Research on mobile machine learning platforms for human gesture recognition in human-machine interaction systems

Downloads

Published

2025-03-28

How to Cite

Barkovska, O., Ruban, I., Tymoshenko, D., Holovchenko, O., & Yankovskyi , O. (2025). Research on mobile machine learning platforms for human gesture recognition in human-machine interaction systems. Technology Audit and Production Reserves, 2(2(82), 6–14. https://doi.org/10.15587/2706-5448.2025.325423

Issue

Vol. 2 No. 2(82) (2025): Information and control systems

Section

Information Technologies

License

Copyright (c) 2025 Olesia Barkovska, Ihor Ruban, Daria Tymoshenko, Oleksandr Holovchenko, Oleksandr Yankovskyi

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.