Benchmarking of transformer-based architectures for fall detection: a comparative study

Authors

DOI:

https://doi.org/10.15587/2706-5448.2025.329398

Keywords:

transformer-based fall detection, temporal convolutional transformer, sensor fusion with barometer data, edge deployment for healthcare AI

Abstract

The object of this research is transformer-oriented deep learning architectures designed for fall detection based on sensor data. One of the main issues identified during the audit of traditional solutions is the excessive computational complexity of standard transformers, which hinders their effective use on resource-constrained devices and in real-time applications. The study involved the use of Temporal Convolutional Transformer, Performer, Multiscale Transformer, LSTM Transformer, Informer, Linformer, and the classical Transformer. Each of these models incorporates advanced mechanisms for attention implementation and processing of both short- and long-term dependencies in input sequences. The Temporal Convolutional Transformer achieved the best results, demonstrating a test accuracy of 99.79% and a peak accuracy of 100% after 50 epochs. This success is attributed to the proposed approach's effective combination of convolutional operations with self-attention, which significantly accelerates the extraction of key features and enables robust handling of short- and long-term temporal dependencies. Convolutional layers help filter out noise from sensor data and reduce computational costs compared to classical transformers. This allows for the deployment of such solutions in real-world edge scenarios without sacrificing fall detection accuracy. Compared to traditional methods, the proposed models offer higher performance and improved resource efficiency – critical factors for implementing real-time fall detection systems. Additionally, the performance of the aforementioned models was evaluated under various operating conditions, including scenarios with low bandwidth and limited energy efficiency. The results confirm that optimized transformer architectures successfully solve the fall detection task while remaining efficient for portable and embedded systems with constrained memory.

Author Biography

Ivan Ursul, Ivan Franko National University of Lviv

PhD Student

Department of Applied Mathematics

References

  1. Gettel, C. J., Chen, K., Goldberg, E. M. (2021). Dementia Care, Fall Detection, and Ambient-Assisted Living Technologies Help Older Adults Age in Place: A Scoping Review. Journal of Applied Gerontology, 40 (12), 1893–1902. https://doi.org/10.1177/07334648211005868
  2. Global, regional, and national burden of diseases and injuries for adults 70 years and older: systematic analysis for the Global Burden of Disease 2019 Study (2022). BMJ, e068208. https://doi.org/10.1136/bmj-2021-068208
  3. Nguyen, H., Mai, T., Nguyen, M. (2024). A Holistic Approach to Elderly Safety: Sensor Fusion, Fall Detection, and Privacy-Preserving Techniques. Image and Video Technology. Singapore: Springer Nature Singapore, 380–393. https://doi.org/10.1007/978-981-97-0376-0_29
  4. Gutiérrez, J., Rodríguez, V., Martin, S. (2021). Comprehensive Review of Vision-Based Fall Detection Systems. Sensors, 21 (3), 947. https://doi.org/10.3390/s21030947
  5. Virginia Anikwe, C., Friday Nweke, H., Chukwu Ikegwu, A., Adolphus Egwuonwu, C., Uchenna Onu, F., Rita Alo, U., Wah Teh, Y. (2022). Mobile and wearable sensors for data-driven health monitoring system: State-of-the-art and future prospect. Expert Systems with Applications, 202, 117362. https://doi.org/10.1016/j.eswa.2022.117362
  6. Choi, H., Um, C. Y., Kang, K., Kim, H., Kim, T. (2021). Review of vision-based occupant information sensing systems for occupant-centric control. Building and Environment, 203, 108064. https://doi.org/10.1016/j.buildenv.2021.108064
  7. Gaya-Morey, F. X., Manresa-Yee, C., Buades-Rubio, J. M. (2024). Deep learning for computer vision based activity recognition and fall detection of the elderly: a systematic review. Applied Intelligence, 54 (19), 8982–9007. https://doi.org/10.1007/s10489-024-05645-1
  8. Ness, S., Eswarakrishnan, V., Sridharan, H., Shinde, V., Venkata Prasad Janapareddy, N., Dhanawat, V. (2025). Anomaly Detection in Network Traffic Using Advanced Machine Learning Techniques. IEEE Access, 13, 16133–16149. https://doi.org/10.1109/access.2025.3526988
  9. Al-Selwi, S. M., Hassan, M. F., Abdulkadir, S. J., Muneer, A. (2023). LSTM Inefficiency in Long-Term Dependencies Regression Problems. Journal of Advanced Research in Applied Sciences and Engineering Technology, 30 (3), 16–31. https://doi.org/10.37934/araset.30.3.1631
  10. Thundiyil, S., Shalamzari, S. S., Picone, J., McKenzie, S. (2023). Transformers for Modeling Long-Term Dependencies in Time Series Data: A Review. 2023 IEEE Signal Processing in Medicine and Biology Symposium (SPMB), 1–5. https://doi.org/10.1109/spmb59478.2023.10372632
  11. Huang, Y., Xu, J., Lai, J., Jiang, Z., Chen, T., Li, Z. et al. (2024). Advancing Transformer architecture in long-context large language models: A comprehensive survey. arXiv. https://doi.org/10.48550/arXiv.2311.12351
  12. Dhanawat, V., Shinde, V., Karande, V., Singhal, K. (2024). Enhancing Financial Risk Management with Federated AI. 2024 8th SLAAI International Conference on Artificial Intelligence (SLAAI-ICAI). Ratmalana 1–6. https://doi.org/10.1109/slaai-icai63667.2024.10844982
  13. Huang, L., Mao, F., Zhang, K., Li, Z. (2022). Spatial-Temporal Convolutional Transformer Network for Multivariate Time Series Forecasting. Sensors, 22 (3), 841. https://doi.org/10.3390/s22030841
  14. Grotowski, J. (2012). Performer. Logiche Della Performance. Accademia University Press, 127–132. https://doi.org/10.4000/books.aaccademia.311
  15. Kim, B., Mun, J., On, K.-W., Shin, M., Lee, J., Kim, E.-S. (2022). Mstr: Multi-scale transformer for end-to-end human-object interaction detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19578–19587. https://doi.org/10.1109/cvpr52688.2022.01897
  16. Xu, M., Xiong, Y., Chen, H. et al. (2021). Long short-term transformer for online action detection. Advances in Neural Information Processing Systems, 34, 1086–1099.
  17. Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., Zhang, W. (2021). Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 35 (12), 11106–11115. https://doi.org/10.1609/aaai.v35i12.17325
  18. Wang, S., Li, B. Z., Khabsa, M., Fang, H., Ma, H. (2020). Linformer: Self-Attention with Linear Complexity. arXiv. https://doi.org/10.48550/arXiv.2006.04768.
  19. Huang, X., Xue, Y., Ren, S., Wang, F. (2023). Sensor-Based Wearable Systems for Monitoring Human Motion and Posture: A Review. Sensors, 23 (22), 9047. https://doi.org/10.3390/s23229047
  20. Bourke, A. K., Lyons, G. M. (2008). A threshold-based fall-detection algorithm using a bi-axial gyroscope sensor. Medical Engineering & Physics, 30 (1), 84–90. https://doi.org/10.1016/j.medengphy.2006.12.001
  21. Minh Dang, L., Min, K., Wang, H., Jalil Piran, Md., Hee Lee, C., Moon, H. (2020). Sensor-based and vision-based human activity recognition: A comprehensive survey. Pattern Recognition, 108, 107561. https://doi.org/10.1016/j.patcog.2020.107561
  22. Kepski, M., Kwolek, B. (2014). Fall detection using ceiling-mounted 3d depth camera. 2014 International conference on computer vision theory and applications (VISAPP), 640–647. https://doi.org/10.5220/0004742406400647
  23. Lin, Z., Wang, Z., Dai, H., Xia, X. (2022). Efficient fall detection in four directions based on smart insoles and RDAE-LSTM model. Expert Systems with Applications, 205, 117661. https://doi.org/10.1016/j.eswa.2022.117661
  24. He, J., Zhang, Q., Wang, L., Pei, L. (2019). Weakly Supervised Human Activity Recognition From Wearable Sensors by Recurrent Attention Learning. IEEE Sensors Journal, 19 (6), 2287–2297. https://doi.org/10.1109/jsen.2018.2885796
  25. Kibet, D., So, M. S., Kang, H., Han, Y., Shin, J.-H. (2024). Sudden Fall Detection of Human Body Using Transformer Model. Sensors, 24 (24), 8051. https://doi.org/10.3390/s24248051
  26. Ursul, I. (2024). Elderly fall detection using unsupervised transformer model. Electronics and Information Technologies, 26. https://doi.org/10.30970/eli.26.7
  27. Haque, S. T., Debnath, M., Yasmin, A., Mahmud, T., Ngu, A. H. H. (2024). Experimental Study of Long Short-Term Memory and Transformer Models for Fall Detection on Smartwatches. Sensors, 24 (19), 6235. https://doi.org/10.3390/s24196235
  28. Haq, I. U., Lee, B. S., Rizzo, D. M. (2024). TransNAS-TSAD: harnessing transformers for multi-objective neural architecture search in time series anomaly detection. Neural Computing and Applications, 37 (4), 2455–2477. https://doi.org/10.1007/s00521-024-10759-1
  29. Ursul, I. (2024). Developing a High-Accuracy Fall Detection Device Using Raspberry Pi and Transformer Models. Ivanursul.com. Available at: https://ivanursul.com/developing-fall-detection-device-raspberry-pi
  30. Ursul, I. (2025). Sensor-Based Fall Detection Dataset with 8,953 Activities from 29 Subjects. Figshare. https://doi.org/10.6084/m9.figshare.28287482.v1
Benchmarking of transformer-based architectures for fall detection: a comparative study

Downloads

Published

2025-05-14

How to Cite

Ursul, I. (2025). Benchmarking of transformer-based architectures for fall detection: a comparative study. Technology Audit and Production Reserves, 3(2(83), 62–70. https://doi.org/10.15587/2706-5448.2025.329398

Issue

Vol. 3 No. 2(83) (2025): Information and control systems

Section

Systems and Control Processes

License

Copyright (c) 2025 Ivan Ursul

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.