Identifying the best models for BISINDO alphabet gesture classification to support the communication needs of the deaf community

Authors

DOI:

https://doi.org/10.15587/1729-4061.2025.332096

Keywords:

BISINDO, alphabet gestures, performance, CNN, VGG-19, MobileNetV2, ResNet-50, Inception-V3

Abstract

The object of this research is the BISINDO alphabet gestures which are static hand movements used by the Deaf community in Indonesia to communicate, with each letter having a unique hand pattern influenced by culture and regional variations. The problem solved is the accuracy and performance of BISINDO hand gesture classification which is less than optimal due to complex gesture variations and computational limitations on lightweight devices.

This study examines the performance of BISINDO alphabet gesture classification using deep learning models with 4 architectures: VGG-19, ResNet-50, MobileNetV2, and Inception-V3. This object is a collection of images used as a dataset consisting of 10,400 images (7,280 training, 2,080 validation, and 1,040 testing). The results show that the MobileNetV2 and VGG-19 architectures achieve the highest accuracy of 100%, followed by Inception-V3 (99%) and ResNet-50 (98%). The results of this study indicate that the superior performance of MobileNetV2 is due to its efficient depthwise separable convolutional architecture, while the superiority of VGG-19 lies in its very deep architecture and the use of small convolutional filters to capture very detailed hierarchical features of gestures. Meanwhile, Inception-V3 excels thanks to the inception module that captures gesture features at various scales. The results of this performance comparison, MobileNetV2 is the most superior because of its computational efficiency that supports high performance, as well as adaptation to complex variations in BISINDO alphabet gestures. The results of this study can also be applied in the fields of education and communication for the deaf community, especially in embedded applications, thus enabling real-time sign language accessibility

Author Biographies

Ilka Zufria, Universitas Islam Negeri Sumatera Utara

Lecturer

Department of Computer Science

Tengku Henny Febriana Harumy, Universitas Sumatera Utara

Lecturer

Department of Computer Science

Syahril Efendi, Universitas Sumatera Utara

Lecturer

Department of Computer Science

References

  1. Types of Hearing Loss. American Speech-Language-Hearing Association. Available at: https://www.asha.org/public/hearing/Types-of-Hearing-Loss
  2. Altiarika, E., Sari, W. P. (2023). Pengembangan Deteksi Realtime untuk Bahasa Isyarat Indonesia dengan Menggunakan Metode Deep Learning Long Short Term Memory dan Convolutional Neural Network. Jurnal Teknologi Informatika Dan Komputer, 9 (1), 1–13. https://doi.org/10.37012/jtik.v9i1.1272
  3. Caraka, R. E., Supardi, K., Kurniawan, R., Kim, Y., Gio, P. U., Yuniarto, B., Mubarok, F. Z., Pardamean, B. (2024). Empowering deaf communication: a novel LSTM model for recognizing Indonesian sign language. Universal Access in the Information Society, 24 (1), 771–783. https://doi.org/10.1007/s10209-024-01095-1
  4. Aziz, A. N. (2021). Image Recognition Alfabet Bahasa Isyarat Indonesia (Bisindo) Menggunakan Metode Convolutional Neural Network. Yogyakarta. Available at: https://dspace.uii.ac.id/handle/123456789/32137
  5. Shams, M. Y., Hassan, E., Gamil, S., Ibrahim, A., Gabr, E., Gamal, S. et al. (2025). Skin Disease Classification: A Comparison of ResNet50, MobileNet, and Efficient-B0. Journal of Current Multidisciplinary Research, 1 (1), 1–7. https://doi.org/10.21608/jcmr.2025.327880.1002
  6. Reka, S. S., Murthy Voona, V. D., Sai Nithish, P. V., Paavan Kumar, D. S., Venugopal, P., Ravi, V. (2023). Performance Analysis of Deep Convolutional Network Architectures for Classification of Over-Volume Vehicles. Applied Sciences, 13 (4), 2549. https://doi.org/10.3390/app13042549
  7. Younis, H., Obaid, M. (2024). Performance Comparison of Pretrained Deep Learning Models for Landfill Waste Classification. International Journal of Advanced Computer Science and Applications, 15 (11). https://doi.org/10.14569/ijacsa.2024.0151166
  8. Sanjaya, S. A., Faustine Ilone, H. (2023). BISINDO Sign Language Recognition: A Systematic Literature Review of Deep Learning Techniques for Image Processing. Indonesian Journal of Computer Science, 12 (6). https://doi.org/10.33022/ijcs.v12i6.3539
  9. Aryananda, I. G. A. O., Samopa, F. (2024). Comparison of the Accuracy of The Bahasa Isyarat Indonesia (BISINDO) Detection System Using CNN and RNN Algorithm for Implementation on Android. MALCOM: Indonesian Journal of Machine Learning and Computer Science, 4 (3), 1111–1119. https://doi.org/10.57152/malcom.v4i3.1465
  10. Pin, K., Ho Chang, J., Nam, Y. (2022). Comparative Study of Transfer Learning Models for Retinal Disease Diagnosis from Fundus Images. Computers, Materials & Continua, 70 (3), 5821–5834. https://doi.org/10.32604/cmc.2022.021943
  11. Cui, S., Su, Y. L., Duan, K., Liu, Y. (2022). Maize leaf disease classification using CBAM and lightweight Autoencoder network. Journal of Ambient Intelligence and Humanized Computing, 14 (6), 7297–7307. https://doi.org/10.1007/s12652-022-04438-z
  12. Kumar, J. S., Anuar, S., Hassan, N. H. (2022). Transfer Learning based Performance Comparison of the Pre-Trained Deep Neural Networks. International Journal of Advanced Computer Science and Applications, 13 (1). https://doi.org/10.14569/ijacsa.2022.0130193
  13. Mendes, J., Lima, J., Costa, L., Rodrigues, N., Pereira, A. I. (2024). Deep learning networks for olive cultivar identification: A comprehensive analysis of convolutional neural networks. Smart Agricultural Technology, 8, 100470. https://doi.org/10.1016/j.atech.2024.100470
  14. Wahyuningsih, W., Nugraha, G. S., Dwiyansaputra, R. (2024). Classification Of Dental Caries Disease In Tooth Images Using A Comparison Of Efficientnet-B0, Mobilenetv2, Resnet-50, Inceptionv3 Architectures. Jurnal Teknik Informatika (Jutif), 5 (4), 177–185. https://doi.org/10.52436/1.jutif.2024.5.4.2187
  15. Candra, A., Rosmalinda, Intan, T. K., Purnamasari, F., Liyanto, H., Nugraha, A. T., Ewaldo. (2024). Development of machine learning-based sign language translator for Bahasa Isyarat Indonesia (BISINDO). Proceedings Of The 6th International Conference On Computing And Applied Informatics 2022, 2987, 020070. https://doi.org/10.1063/5.0199747
  16. Harumy, T. H. F., Br Ginting, D. S., Manik, F. Y., Alkhowarizmi, A. (2024). Developing an early detection model for skin diseases using a hybrid deep neural network to enhance health independence in coastal communities. Eastern-European Journal of Enterprise Technologies, 6 (9 (132)), 71–85. https://doi.org/10.15587/1729-4061.2024.313983
  17. Díaz-Gaxiola, E., Morales-Casas, Z. E., Castro-López, O., Beltrán-Gutiérrez, G., Vega-López, I. F., Yee-Rendón, A. (2019). Estudio comparativo de arquitecturas de CNNs en hojas de Pimiento Morrón infectadas con virus PHYVV o PEPGMV. Research in Computing Science, 148 (7), 289–303. https://doi.org/10.13053/rcs-148-7-22
  18. Kunjachan, S., Remya, R., Kumar, S., Asish, G. R., Sheela, K., Kala, S. (2023). Comparative study of convolutional neural networks for leaf classification in Ayurveda. IET Conference Proceedings, 2023 (11), 18–23. https://doi.org/10.1049/icp.2023.1756
  19. Iparraguirre-Villanueva, O., Guevara-Ponce, V., Paredes, O. R., Sierra-Liñan, F., Zapata-Paulini, J., Cabanillas-Carbonell, M. (2022). Convolutional Neural Networks with Transfer Learning for Pneumonia Detection. International Journal of Advanced Computer Science and Applications, 13 (9). https://doi.org/10.14569/ijacsa.2022.0130963
  20. Panthakkan, A., Anzar, S. M., Al Mansoori, S., Mansoor, W., Al Ahmad, H. (2022). A systematic comparison of transfer learning models for COVID-19 prediction. Intelligent Decision Technologies, 16 (3), 557–574. https://doi.org/10.3233/idt-220017
  21. Ahmed, N., Rahman, M., Ishrak, F., Joy, I. K., Sabuj, S. H., Rahman, S. (2024). Comparative Performance Analysis of Transformer-Based Pre-Trained Models for Detecting Keratoconus Disease. arXiv. https://doi.org/10.48550/arXiv.2408.09005
  22. Ahmad, N., Wijaya, E. S., Tjoaquinn, C., Lucky, H., Iswanto, I. A. (2023). Transforming Sign Language using CNN Approach based on BISINDO Dataset. 2023 International Conference on Informatics, Multimedia, Cyber and Informations System (ICIMCIS), 543–548. https://doi.org/10.1109/icimcis60089.2023.10349011
  23. Hossain, Md. B., Iqbal, S. M. H. S., Islam, Md. M., Akhtar, Md. N., Sarker, I. H. (2022). Transfer learning with fine-tuned deep CNN ResNet50 model for classifying COVID-19 from chest X-ray images. Informatics in Medicine Unlocked, 30, 100916. https://doi.org/10.1016/j.imu.2022.100916
  24. Agarwal, C., Vishwakarma, V. P. (2022). Comparison of Different Deep CNN Models for Leukemia Diagnosis. Proceedings of the International Conference on Cognitive and Intelligent Computing, 659–672. https://doi.org/10.1007/978-981-19-2350-0_63
  25. Gulzar, Y. (2023). Fruit Image Classification Model Based on MobileNetV2 with Deep Transfer Learning Technique. Sustainability, 15 (3), 1906. https://doi.org/10.3390/su15031906
  26. Indraswari, R., Rokhana, R., Herulambang, W. (2022). Melanoma image classification based on MobileNetV2 network. Procedia Computer Science, 197, 198–207. https://doi.org/10.1016/j.procs.2021.12.132
  27. Theckedath, D., Sedamkar, R. R. (2020). Detecting Affect States Using VGG16, ResNet50 and SE-ResNet50 Networks. SN Computer Science, 1 (2). https://doi.org/10.1007/s42979-020-0114-9
  28. Cai, W., Li, M., Jin, G., Liu, Q., Lu, C. (2024). Comparison of Residual Network and Other Classical Models for Classification of Interlayer Distresses in Pavement. Applied Sciences, 14 (15), 6568. https://doi.org/10.3390/app14156568
  29. Patra, P., Singh, T. (2022). Diabetic Retinopathy Detection using an Improved ResNet 50-InceptionV3 and hybrid DiabRetNet Structures. 2022 OITS International Conference on Information Technology (OCIT), 140–145. https://doi.org/10.1109/ocit56763.2022.00036
  30. Wang, C., Chen, D., Hao, L., Liu, X., Zeng, Y., Chen, J., Zhang, G. (2019). Pulmonary Image Classification Based on Inception-v3 Transfer Learning Model. IEEE Access, 7, 146533–146541. https://doi.org/10.1109/access.2019.2946000
  31. Wang, X., Li, J., Tao, J., Wu, L., Mou, C., Bai, W. et al. (2022). A Recognition Method of Ancient Architectures Based on the Improved Inception V3 Model. Symmetry, 14 (12), 2679. https://doi.org/10.3390/sym14122679
  32. Harumy, T. H. F., Zarlis, M., Lydia, M. S., Efendi, S. (2024). Image classification of diseases in rice using deep neural neural network and incevtion V3. Proceedings Of The 6th International Conference On Computing And Applied Informatics 2022, 2987, 020046. https://doi.org/10.1063/5.0200203
Identifying the best models for BISINDO alphabet gesture classification to support the communication needs of the deaf community

Downloads

Published

2025-12-31

How to Cite

Zufria, I., Harumy, T. H. F., & Efendi, S. (2025). Identifying the best models for BISINDO alphabet gesture classification to support the communication needs of the deaf community. Eastern-European Journal of Enterprise Technologies, 6(2 (138), 26–41. https://doi.org/10.15587/1729-4061.2025.332096

Issue

Vol. 6 No. 2 (138) (2025): Information technology. Industry control systems

Section

Information technology

License

Copyright (c) 2025 Ilka Zufria, Tengku Henny Febriana Harumy, Syahril Efendi

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.

A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.