Construction of a neural network for handwritten digits recognition based on TensorFlow library applying an error backpropagation algorithm

Authors

DOI:

https://doi.org/10.15587/1729-4061.2023.293682

Keywords:

neural network, loss function, gradient descent, neural network accuracy

Abstract

The object of this study is a neural network for recognizing handwritten digits based on the TensorFlow library using the backpropagation algorithm.

The main problem addressed is the development of an effective model with high recognition accuracy. Working on such a task is important as it allows understanding how algorithms and models can effectively work with real data and helps improve machine learning techniques.

It has been determined that after 20 training epochs, the loss function is 0.105, and the recognition accuracy is 0.976, comparable to human recognition capability. The classification report indicates that the model is effectively trained on training data and demonstrates high accuracy on test data, capable of generalizing information to new examples. Visualization of recognition results confirms that the model correctly recognizes even poorly written digits.

The results can be explained by the peculiarities of the model architecture, optimal selection of hyperparameters, and successful use of the backpropagation algorithm, which was not explicitly specified during model training. TensorFlow provided a convenient toolkit for implementing the neural network and optimizing its parameters. As a result, the model has a fairly high accuracy in image recognition.

A significant feature of the results is the high recognition accuracy achieved through the optimal model architecture, correct choice of hyperparameters, and effective use of the backpropagation algorithm. Unlike models built using Keras and convolutional layers, the research model quickly learns, which is important, and does not compromise on accuracy. This result was made possible by the above features of model construction.

The results could be practically applied in the field of handwritten character recognition, especially in automated document classification systems, in banking recognition systems, and in other areas where the accuracy of handwritten character recognition is essential

Author Biographies

Tetiana Filimonova, State University of Trade and Economics

PhD

Department of Computer Science and Information Systems

Hanna Samoylenko, State University of Trade and Economics

PhD, Associate Professor

Department of Computer Science and Information Systems

Anna Selivanova, State University of Trade and Economics

Department of Computer Science and Information Systems

Yurii Yurchenko, State University of Trade and Economics

Department of Computer Science and Information Systems

Alexei Parashchak, State University of Trade and Economics

PhD

Department of Computer Science and Information Systems

References

  1. Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A. (2014). Return of the Devil in the Details: Delving Deep into Convolutional Nets. Proceedings of the British Machine Vision Conference 2014. doi: https://doi.org/10.5244/c.28.6
  2. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S. et al. (2015). ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 115 (3), 211–252. doi: https://doi.org/10.1007/s11263-015-0816-y
  3. Tuba, E., Tuba, M., Simian, D. (2016). Handwritten digit recognition by support vector machine optimized by bat algorithm. WSCG 2016 - 24th Conference on Computer Graphics, Visualization and Computer Vision 2016, 369–376. Available at: https://dspace5.zcu.cz/bitstream/11025/29725/1/Tuba.pdf
  4. Elleuch, M., Maalej, R., Kherallah, M. (2016). A New Design Based-SVM of the CNN Classifier Architecture with Dropout for Offline Arabic Handwritten Recognition. Procedia Computer Science, 80, 1712–1723. doi: https://doi.org/10.1016/j.procs.2016.05.512
  5. Reshma, A. J., James, J. J., Kavya, M., Saravanan, M. (2016). An overview of character recognition focused on offline handwriting. ARPN Journal of Engineering and Applied Sciences, 11 (15), 9372–9378. Available at: http://www.arpnjournals.org/jeas/research_papers/rp_2016/jeas_0816_4774.pdf
  6. Alom, M. Z., Sidike, P., Taha, T. M., Asari, V. K. (2017). Handwritten bangla digit recognition using deep learning. arXiv. doi: https://doi.org/10.48550/arXiv.1705.02680
  7. Maitra, D. S., Bhattacharya, U., Parui, S. K. (2015). CNN based common approach to handwritten character recognition of multiple scripts. 2015 13th International Conference on Document Analysis and Recognition (ICDAR). doi: https://doi.org/10.1109/icdar.2015.7333916
  8. Glauner, P. O. (2015). Comparison of training methods for deep neural networks. arXiv. doi: https://doi.org/10.48550/arXiv.1504.06825
  9. Guerra, L., McGarry, L. M., Robles, V., Bielza, C., Larrañaga, P., Yuste, R. (2010). Comparison between supervised and unsupervised classifications of neuronal cell types: A case study. Developmental Neurobiology, 71 (1), 71–82. doi: https://doi.org/10.1002/dneu.20809
  10. Stenin, A., Pasko, V., Soldatova, M., Drozdovich, I. (2022). Recognition of handwritten numbers on the basis of convolutional neural networks. Adaptive systems of automatic control, 2 (41), 39–44. Available at: http://asac.kpi.ua/article/view/271337/
  11. Korotun, O. V., Marchuk, H. V., Marchuk, D. K., Talaver, O. V. (2020). Systema rozpiznavannia rukopysnykh tsyfr z otsinkoiu yakosti. Tekhnichna inzheneriya, 1 (85), 135–146. doi: https://doi.org/10.26642/ten-2020-1(85)-135-146
  12. Kraskovska, A. O., Filimonova, T. O. (2023). Rozrobka arkhitektury zghortkovoi neironnoi merezhi dlia rozpiznavannia rukopysnykh tsyfr. Zbirnyk tez XX mizhnarodnoi naukovo-praktychnoi konferentsiyi «Matematychne na prohramne zabezpechennia intelektualnykh system. MPZIS – 2023». Dnipro, 165–167.
  13. El-Sawy, A., EL-Bakry, H., Loey, M. (2016). CNN for Handwritten Arabic Digits Recognition Based on LeNet-5. Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2016, 566–575. doi: https://doi.org/10.1007/978-3-319-48308-5_54
  14. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 (11), 2278–2324. doi: https://doi.org/10.1109/5.726791
  15. Whittington, J. C. R., Bogacz, R. (2019). Theories of Error Back-Propagation in the Brain. Trends in Cognitive Sciences, 23 (3), 235–250. doi: https://doi.org/10.1016/j.tics.2018.12.005
  16. Whittington, J. C. R., Bogacz, R. (2017). An approximation of the error backpropagation algorithm in a predictive coding network with local hebbian synaptic plasticity. Neural Computation, 29 (5), 1229–1262. doi: https://doi.org/10.1162/neco_a_00949
  17. Layer activation functions. Available at: https://keras.io/api/layers/activations/
  18. Datasets. MNIST. TensorFlow. Available at: https://www.tensorflow.org/datasets/catalog/mnist
  19. Podoliak, B. Yu., Filimonova, T. O. (2023). Rozrobka avtokoduvalnyka dlia rozpiznavannia rukopysnykh tsyfr. Zbirnyk tez XX mizhnarodnoi naukovo-praktychnoi konferentsiyi «Matematychne na prohramne zabezpechennia intelektualnykh system. MPZIS – 2023». Dnipro, 243–244.
Construction of a neural network for handwritten digits recognition based on TensorFlow library applying an error backpropagation algorithm

Downloads

Published

2023-12-29

How to Cite

Filimonova, T., Samoylenko, H., Selivanova, A., Yurchenko, Y., & Parashchak, A. (2023). Construction of a neural network for handwritten digits recognition based on TensorFlow library applying an error backpropagation algorithm. Eastern-European Journal of Enterprise Technologies, 6(2 (126), 25–32. https://doi.org/10.15587/1729-4061.2023.293682

Issue

Vol. 6 No. 2 (126) (2023): Information technology. Industry control systems

Section

Information technology

License

Copyright (c) 2023 Tetiana Filimonova, Hanna Samoylenko, Anna Selivanova, Yurii Yurchenko, Alexei Parashchak

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.

A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.