Building a mathematical model and an algorithm for training a neural network with sparse dipole synaptic connections for image recognition
DOI:
https://doi.org/10.15587/1729-4061.2021.245010Keywords:
mathematical model, neural network, sparse dipole synaptic connections, image recognitionAbstract
Large enough structured neural networks are used for solving the tasks to recognize distorted images involving computer systems. One such neural network that can completely restore a distorted image is a fully connected pseudospin (dipole) neural network that possesses associative memory. When submitting some image to its input, it automatically selects and outputs the image that is closest to the input one. This image is stored in the neural network memory within the Hopfield paradigm. Within this paradigm, it is possible to memorize and reproduce arrays of information that have their own internal structure.
In order to reduce learning time, the size of the neural network is minimized by simplifying its structure based on one of the approaches: underlying the first is «regularization» while the second is based on the removal of synaptic connections from the neural network. In this work, the simplification of the structure of a fully connected dipole neural network is based on the dipole-dipole interaction between the nearest adjacent neurons of the network.
It is proposed to minimize the size of a neural network through dipole-dipole synaptic connections between the nearest neurons, which reduces the time of the computational resource in the recognition of distorted images. The ratio for weight coefficients of synaptic connections between neurons in dipole approximation has been derived. A training algorithm has been built for a dipole neural network with sparse synaptic connections, which is based on the dipole-dipole interaction between the nearest neurons. A computer experiment was conducted that showed that the neural network with sparse dipole connections recognizes distorted images 3 times faster (numbers from 0 to 9, which are shown at 25 pixels), compared to a fully connected neural network
References
Peleshchak, I., Peleshchak, R., Lytvyn, V., Kopka, J., Wrzesien, M., Korniak, J. et. al. (2020). Spectral Image Recognition Using Artificial Dynamic Neural Network in Information Resonance Mode. Artificial Intelligence and Industrial Applications, 313–322. doi: https://doi.org/10.1007/978-3-030-51186-9_22
Lytvyn, V., Peleshchak, I., Peleshchak, R., Holoshchuk, R. (2018). Detection of multispectral input images using nonlinear artificial neural networks. 2018 14th International Conference on Advanced Trends in Radioelecrtronics, Telecommunications and Computer Engineering (TCSET). doi: https://doi.org/10.1109/tcset.2018.8336169
Greenberg, S., Guterman, H. (1996). Neural-network classifiers for automatic real-world aerial image recognition. Applied Optics, 35 (23), 4598. doi: https://doi.org/10.1364/ao.35.004598
Andriyanov, N. A., Dementiev, V. E., Kargashin, Y. D. (2021). Analysis of the impact of visual attacks on the characteristics of neural networks in image recognition. Procedia Computer Science, 186, 495–502. doi: https://doi.org/10.1016/j.procs.2021.04.170
Simard, P. Y., Steinkraus, D., Platt, J. C. (2003). Best practices for convolutional neural networks applied to visual document analysis. Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings. doi: https://doi.org/10.1109/icdar.2003.1227801
Zhou, Y., Song, S., Cheung, N.-M. (2017). On classification of distorted images with deep convolutional neural networks. 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). doi: https://doi.org/10.1109/icassp.2017.7952349
Ha, M., Byun, Y., Kim, J., Lee, J., Lee, Y., Lee, S. (2019). Selective Deep Convolutional Neural Network for Low Cost Distorted Image Classification. IEEE Access, 7, 133030–133042. doi: https://doi.org/10.1109/access.2019.2939781
Li, B., Tian, M., Zhang, W., Yao, H., Wang, X. (2021). Learning to predict the quality of distorted-then-compressed images via a deep neural network. Journal of Visual Communication and Image Representation, 76, 103004. doi: https://doi.org/10.1016/j.jvcir.2020.103004
Guan, X., Li, F., He, L. (2020). Quality Assessment on Authentically Distorted Images by Expanding Proxy Labels. Electronics, 9 (2), 252. doi: https://doi.org/10.3390/electronics9020252
Peleshchak, R., Lytvyn, V., Peleshchak, I., Vysotska, V. (2021). Stochastic Pseudo-Spin Neural Network with Tridiagonal Synaptic Connections. 2021 IEEE International Conference on Smart Information Systems and Technologies (SIST). doi: https://doi.org/10.1109/sist50301.2021.9465998
Slyadnikov, E. E. (2007). Fizicheskaya model' i associativnaya pamyat' dipol'noy sistemy mikrotrubochki citoskeleta. Zhurnal tehnicheskoy fiziki, 77 (7), 77–86. Availale at: https://journals.ioffe.ru/articles/viewPDF/9173
Slyadnikov, E. E. (2011). Fizicheskie osnovy, modeli predstavleniya i raspoznavaniya obrazov v mikrotrubochke citoskeleta neyrona. Zhurnal tehnicheskoy fiziki, 81 (12). Availale at: http://journals.ioffe.ru/articles/viewPDF/10478
Penrouz, R. (2005). Teni razuma: v poiskah nauki o soznanii. Moscow-Izhevsk: IKI, 688. Availale at: http://alpha.sinp.msu.ru/~panov/Penrose-Shadows.pdf
Hameroff, S. R. (1994). Quantum coherence in microtubules: A neural basis for emergent consciousness? Journal of Consciousness Studies, 1 (1), 91–118. Availale at: https://www.ingentaconnect.com/contentone/imp/jcs/1994/00000001/00000001/art00008
Brown, J. A., Tuszynski, J. A. (1999). A review of the ferroelectric model of microtubules. Ferroelectrics, 220 (1), 141–155. doi: https://doi.org/10.1080/00150199908216213
Tuszyński, J. A., Hameroff, S., Satarić, M. V., Trpisová, B., Nip, M. L. A. (1995). Ferroelectric behavior in microtubule dipole lattices: Implications for information processing, signaling and assembly/disassembly. Journal of Theoretical Biology, 174 (4), 371–380. doi: https://doi.org/10.1006/jtbi.1995.0105
Stebbings, H. (1995). Microtubule-based intracellular transport of organelles. The Cytoskeleton: A Multi-Volume Treatise, 113–140. doi: https://doi.org/10.1016/s1874-6020(06)80017-0
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79 (8), 2554–2558. doi: https://doi.org/10.1073/pnas.79.8.2554
Yurkovych, N. V., Herasimov, O. V., Yurkovych, V. M., Mar’yan, M. I. (2014). Composition of neural networks by hebb algorithm and direct spreading in characters encoding systems. Uzhhorod University Scientific Herald. Series Physics, 36, 161–167. Availale at: http://teib.info/?wpfb_dl=1138
Chernіak, O., Peleshchak, R., Doroshenko, M. (2020). Reduction of display time of input images by pseudo-spin neural network due to rarefaction of synaptic connections. Modern problems in science. Abstracts of VIII International Scientific and Practical Conference. Prague, 680–686. Availale at: https://isg-konf.com/uk/modern-problems-in-science-ua/
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Vasyl Lytvyn, Roman Peleshchak, Ivan Peleshchak, Oksana Cherniak, Lyubomyr Demkiv
This work is licensed under a Creative Commons Attribution 4.0 International License.
The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.
