Building a mathematical model and an algorithm for training a neural network with sparse dipole synaptic connections for image recognition

Authors

DOI:

https://doi.org/10.15587/1729-4061.2021.245010

Keywords:

mathematical model, neural network, sparse dipole synaptic connections, image recognition

Abstract

Large enough structured neural networks are used for solving the tasks to recognize distorted images involving computer systems. One such neural network that can completely restore a distorted image is a fully connected pseudospin (dipole) neural network that possesses associative memory. When submitting some image to its input, it automatically selects and outputs the image that is closest to the input one. This image is stored in the neural network memory within the Hopfield paradigm. Within this paradigm, it is possible to memorize and reproduce arrays of information that have their own internal structure.

In order to reduce learning time, the size of the neural network is minimized by simplifying its structure based on one of the approaches: underlying the first is «regularization» while the second is based on the removal of synaptic connections from the neural network. In this work, the simplification of the structure of a fully connected dipole neural network is based on the dipole-dipole interaction between the nearest adjacent neurons of the network.

It is proposed to minimize the size of a neural network through dipole-dipole synaptic connections between the nearest neurons, which reduces the time of the computational resource in the recognition of distorted images. The ratio for weight coefficients of synaptic connections between neurons in dipole approximation has been derived. A training algorithm has been built for a dipole neural network with sparse synaptic connections, which is based on the dipole-dipole interaction between the nearest neurons. A computer experiment was conducted that showed that the neural network with sparse dipole connections recognizes distorted images 3 times faster (numbers from 0 to 9, which are shown at 25 pixels), compared to a fully connected neural network

Author Biographies

Vasyl Lytvyn, Lviv Polytechnic National University

Doctor of Technical Sciences, Professor

Department of Information Systems and Networks

Roman Peleshchak, Lviv Polytechnic National University

Doctor of Physical and Mathematical Sciences, Professor

Department of Information Systems and Networks

Ivan Peleshchak, Lviv Polytechnic National University

Postgraduate Student

Department of Information Systems and Networks

Oksana Cherniak, Drohobych Ivan Franko State Pedagogical University

Postgraduate Student

Department of Mathematics

Lyubomyr Demkiv, Lviv Polytechnic National University

Doctor of Technical Sciences, Professor

Department of Information Systems and Networks

References

Peleshchak, I., Peleshchak, R., Lytvyn, V., Kopka, J., Wrzesien, M., Korniak, J. et. al. (2020). Spectral Image Recognition Using Artificial Dynamic Neural Network in Information Resonance Mode. Artificial Intelligence and Industrial Applications, 313–322. doi: https://doi.org/10.1007/978-3-030-51186-9_22

Lytvyn, V., Peleshchak, I., Peleshchak, R., Holoshchuk, R. (2018). Detection of multispectral input images using nonlinear artificial neural networks. 2018 14th International Conference on Advanced Trends in Radioelecrtronics, Telecommunications and Computer Engineering (TCSET). doi: https://doi.org/10.1109/tcset.2018.8336169

Greenberg, S., Guterman, H. (1996). Neural-network classifiers for automatic real-world aerial image recognition. Applied Optics, 35 (23), 4598. doi: https://doi.org/10.1364/ao.35.004598

Andriyanov, N. A., Dementiev, V. E., Kargashin, Y. D. (2021). Analysis of the impact of visual attacks on the characteristics of neural networks in image recognition. Procedia Computer Science, 186, 495–502. doi: https://doi.org/10.1016/j.procs.2021.04.170

Simard, P. Y., Steinkraus, D., Platt, J. C. (2003). Best practices for convolutional neural networks applied to visual document analysis. Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings. doi: https://doi.org/10.1109/icdar.2003.1227801

Zhou, Y., Song, S., Cheung, N.-M. (2017). On classification of distorted images with deep convolutional neural networks. 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). doi: https://doi.org/10.1109/icassp.2017.7952349

Ha, M., Byun, Y., Kim, J., Lee, J., Lee, Y., Lee, S. (2019). Selective Deep Convolutional Neural Network for Low Cost Distorted Image Classification. IEEE Access, 7, 133030–133042. doi: https://doi.org/10.1109/access.2019.2939781

Li, B., Tian, M., Zhang, W., Yao, H., Wang, X. (2021). Learning to predict the quality of distorted-then-compressed images via a deep neural network. Journal of Visual Communication and Image Representation, 76, 103004. doi: https://doi.org/10.1016/j.jvcir.2020.103004

Guan, X., Li, F., He, L. (2020). Quality Assessment on Authentically Distorted Images by Expanding Proxy Labels. Electronics, 9 (2), 252. doi: https://doi.org/10.3390/electronics9020252

Peleshchak, R., Lytvyn, V., Peleshchak, I., Vysotska, V. (2021). Stochastic Pseudo-Spin Neural Network with Tridiagonal Synaptic Connections. 2021 IEEE International Conference on Smart Information Systems and Technologies (SIST). doi: https://doi.org/10.1109/sist50301.2021.9465998

Slyadnikov, E. E. (2007). Fizicheskaya model' i associativnaya pamyat' dipol'noy sistemy mikrotrubochki citoskeleta. Zhurnal tehnicheskoy fiziki, 77 (7), 77–86. Availale at: https://journals.ioffe.ru/articles/viewPDF/9173

Slyadnikov, E. E. (2011). Fizicheskie osnovy, modeli predstavleniya i raspoznavaniya obrazov v mikrotrubochke citoskeleta neyrona. Zhurnal tehnicheskoy fiziki, 81 (12). Availale at: http://journals.ioffe.ru/articles/viewPDF/10478

Penrouz, R. (2005). Teni razuma: v poiskah nauki o soznanii. Moscow-Izhevsk: IKI, 688. Availale at: http://alpha.sinp.msu.ru/~panov/Penrose-Shadows.pdf

Hameroff, S. R. (1994). Quantum coherence in microtubules: A neural basis for emergent consciousness? Journal of Consciousness Studies, 1 (1), 91–118. Availale at: https://www.ingentaconnect.com/contentone/imp/jcs/1994/00000001/00000001/art00008

Brown, J. A., Tuszynski, J. A. (1999). A review of the ferroelectric model of microtubules. Ferroelectrics, 220 (1), 141–155. doi: https://doi.org/10.1080/00150199908216213

Tuszyński, J. A., Hameroff, S., Satarić, M. V., Trpisová, B., Nip, M. L. A. (1995). Ferroelectric behavior in microtubule dipole lattices: Implications for information processing, signaling and assembly/disassembly. Journal of Theoretical Biology, 174 (4), 371–380. doi: https://doi.org/10.1006/jtbi.1995.0105

Stebbings, H. (1995). Microtubule-based intracellular transport of organelles. The Cytoskeleton: A Multi-Volume Treatise, 113–140. doi: https://doi.org/10.1016/s1874-6020(06)80017-0

Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79 (8), 2554–2558. doi: https://doi.org/10.1073/pnas.79.8.2554

Yurkovych, N. V., Herasimov, O. V., Yurkovych, V. M., Mar’yan, M. I. (2014). Composition of neural networks by hebb algorithm and direct spreading in characters encoding systems. Uzhhorod University Scientific Herald. Series Physics, 36, 161–167. Availale at: http://teib.info/?wpfb_dl=1138

Chernіak, O., Peleshchak, R., Doroshenko, M. (2020). Reduction of display time of input images by pseudo-spin neural network due to rarefaction of synaptic connections. Modern problems in science. Abstracts of VIII International Scientific and Practical Conference. Prague, 680–686. Availale at: https://isg-konf.com/uk/modern-problems-in-science-ua/

Published

2021-12-16

How to Cite

Lytvyn, V., Peleshchak, R., Peleshchak, I., Cherniak, O., & Demkiv, L. (2021). Building a mathematical model and an algorithm for training a neural network with sparse dipole synaptic connections for image recognition. Eastern-European Journal of Enterprise Technologies, 6(4 (114), 21–27. https://doi.org/10.15587/1729-4061.2021.245010

Issue

Section

Mathematics and Cybernetics - applied aspects