Improvement of the model of object recognition in aero photographs using deep convolutional neural networks

Authors

DOI:

https://doi.org/10.15587/1729-4061.2021.243094

Keywords:

object recognition, deep convolutional neural network, aerial photograph, unmanned aerial vehicle

Abstract

Detection and recognition of objects in images is the main problem to be solved by computer vision systems. As part of solving this problem, the model of object recognition in aerial photographs taken from unmanned aerial vehicles has been improved. A study of object recognition in aerial photographs using deep convolutional neural networks has been carried out. Analysis of possible implementations showed that the AlexNet 2012 model (Canada) trained on the ImageNet image set (China) is most suitable for this problem solution. This model was used as a basic one. The object recognition error for this model with the use of the ImageNet test set of images amounted to 15 %. To solve the problem of improving the effectiveness of object recognition in aerial photographs for 10 classes of images, the final fully connected layer was modified by rejection from 1,000 to 10 neurons and additional two-stage training of the resulting model. Additional training was carried out with a set of images prepared from aerial photographs at stage 1 and with a set of VisDrone 2021 (China) images at stage 2. Optimal training parameters were selected: speed (step) (0.0001), number of epochs (100). As a result, a new model under the proposed name of AlexVisDrone was obtained.

The effectiveness of the proposed model was checked with a test set of 100 images for each class (the total number of classes was 10). Accuracy and sensitivity were chosen as the main indicators of the model effectiveness. As a result, an increase in recognition accuracy from 7 % (for images from aerial photographs) to 9 % (for the VisDrone 2021 set) was obtained which has indicated that the choice of neural network architecture and training parameters was correct. The use of the proposed model makes it possible to automate the process of object recognition in aerial photographs.

In the future, it is advisable to use this model at ground stations of unmanned aerial vehicle complex control when processing aerial photographs taken from unmanned aerial vehicles, in robotic systems, in video surveillance complexes and when designing unmanned vehicle systems

Author Biographies

Vadym Slyusar, Central Scientific Research Institute of the Army of the Armed Forces of Ukraine

Doctor of Technical Sciences, Professor

Research Institute Group

Mykhailo Protsenko, Central Scientific Research Institute of the Army of the Armed Forces of Ukraine

PhD, Senior Researcher

Office of Special Forces

Anton Chernukha, National University of Civil Defence of Ukraine

PhD

Department of Fire and Rescue Training

Pavlo Kovalov, National University of Civil Defence of Ukraine

PhD, Associate Professor

Department of Fire and Rescue Training

Pavlo Borodych, National University of Civil Defence of Ukraine

PhD, Associate Professor

Department of Fire and Rescue Training

Serhii Shevchenko, National University of Civil Defence of Ukraine

PhD

Department of Fire Tactics and Rescue Operations

Oleksandr Chernikov, Kharkiv National Automobile and Highway University

Doctor of Technical Sciences, Professor

Department of Engineering and Computer Graphics

Serhii Vazhynskyi, Ivan Kozhedub Kharkiv National Air Force University

PhD, Associate Professor

Scientific Center Air Force

Oleg Bogatov, Kharkiv National Automobile and Highway University

PhD, Associate Professor

Department of Metrology and Life Safety

Kirill Khrustalev, Kharkiv National University of Radio Electronics

PhD, Associate Professor

Department of Computer-Integrated Technologies, Automation and Mechatronics

References

  1. Yang, X., Lin, D., Zhang, F., Song, T., Jiang, T. (2019). High Accuracy Active Stand-off Target Geolocation Using UAV Platform. 2019 IEEE International Conference on Signal, Information and Data Processing (ICSIDP). doi: http://doi.org/10.1109/icsidp47821.2019.9172919
  2. Pospelov, B., Andronov, V., Rybka, E., Krainiukov, O., Maksymenko, N., Meleshchenko, R. et. al. (2020). Mathematical model of determining a risk to the human health along with the detection of hazardous states of urban atmosphere pollution based on measuring the current concentrations of pollutants. Eastern-European Journal of Enterprise Technologies, 4 (10 (106)), 37–44. doi: http://doi.org/10.15587/1729-4061.2020.210059
  3. Chernukha, A., Teslenko, A., Kovalov, P., Bezuglov, O. (2020). Mathematical Modeling of Fire-Proof Efficiency of Coatings Based on Silicate Composition. Materials Science Forum, 1006, 70–75. doi: http://doi.org/10.4028/www.scientific.net/msf.1006.70
  4. Vambol, S., Vambol, V., Kondratenko, O., Suchikova, Y., Hurenko, O. (2017). Assessment of improvement of ecological safety of power plants by arranging the system of pollutant neutralization. Eastern-European Journal of Enterprise Technologies, 3 (10 (87)), 63–73. doi: http://doi.org/10.15587/1729-4061.2017.102314
  5. Vambol, S., Vambol, V., Kondratenko, O., Koloskov, V., Suchikova, Y. (2018). Substantiation of expedience of application of high-temperature utilization of used tires for liquefied methane production. Journal of Achievements in Materials and Manufacturing Engineering, 2 (87), 77–84. doi: http://doi.org/10.5604/01.3001.0012.2830
  6. Teslenko, A., Chernukha, A., Bezuglov, O., Bogatov, O., Kunitsa, E., Kalyna, V. et. al. (2019). Construction of an algorithm for building regions of questionable decisions for devices containing gases in a linear multidimensional space of hazardous factors. Eastern-European Journal of Enterprise Technologies, 5 (10 (101)), 42–49. doi: http://doi.org/10.15587/1729-4061.2019.181668
  7. Semko, A., Rusanova, O., Kazak, O., Beskrovnaya, M., Vinogradov, S., Gricina, I. (2015). The use of pulsed high-speed liquid jet for putting out gas blow-out. The International Journal of Multiphysics, 9 (1), 9–20. doi: http://doi.org/10.1260/1750-9548.9.1.9
  8. Pospelov, B., Andronov, V., Rybka, E., Popov, V., Semkiv, O. (2018). Development of the method of frequency­temporal representation of fluctuations of gaseous medium parameters at fire. Eastern-European Journal of Enterprise Technologies, 2 (10 (92)), 44–49. doi: http://doi.org/10.15587/1729-4061.2018.125926
  9. Pospelov, B., Rybka, E., Meleshchenko, R., Borodych, P., Gornostal, S. (2019). Development of the method for rapid detection of hazardous atmospheric pollution of cities with the help of recurrence measures. Eastern-European Journal of Enterprise Technologies, 1 (10 (97)), 29–35. doi: http://doi.org/10.15587/1729-4061.2019.155027
  10. Slyusar, V., Protsenko, M., Chernukha, A., Gornostal, S., Rudakov, S., Shevchenko, S. et. al. (2021). Construction of an advanced method for recognizing monitored objects by a convolutional neural network using a discrete wavelet transform. Eastern-European Journal of Enterprise Technologies, 4 (9 (112)), 65–77. doi: http://doi.org/10.15587/1729-4061.2021.238601
  11. Sermanet, P., Kavukcuoglu, K., Chintala, S., Lecun, Y. (2013). Pedestrian Detection with Unsupervised Multi-stage Feature Learning. 2013 IEEE Conference on Computer Vision and Pattern Recognition, 3626–3633. doi: http://doi.org/10.1109/cvpr.2013.465
  12. Lu, X., Ma, C., Ni, B., Yang, X. (2021). Adaptive Region Proposal With Channel Regularization for Robust Object Tracking. IEEE Transactions on Circuits and Systems for Video Technology, 31 (4), 1268–1282. doi: http://doi.org/10.1109/tcsvt.2019.2944654
  13. Li, J., Weinmann, M., Sun, X., Diao, W., Feng, Y., Fu, K. (2021). Random Topology and Random Multiscale Mapping: An Automated Design of Multiscale and Lightweight Neural Network for Remote-Sensing Image Recognition. IEEE Transactions on Geoscience and Remote Sensing, 1–17. doi: http://doi.org/10.1109/tgrs.2021.3102988
  14. Zhang, Z., Zhang, L., Wang, Y., Feng, P., He, R. (2021). ShipRSImageNet: A Large-Scale Fine-Grained Dataset for Ship Detection in High-Resolution Optical Remote Sensing Images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 8458–8472. doi: http://doi.org/10.1109/jstars.2021.3104230
  15. Fan, J., Lee, J., Lee, Y. (2021). Image Classification Using Fusion of Multiple Neural Networks. 2021 36th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC). doi: http://doi.org/10.1109/itc-cscc52171.2021.9501468
  16. Wu, C., Shao, S., Tunc, C., Hariri, S. (2020). Video Anomaly Detection using Pre-Trained Deep Convolutional Neural Nets and Context Mining. 2020 IEEE/ACS 17th International Conference on Computer Systems and Applications (AICCSA). doi: http://doi.org/10.1109/aiccsa50499.2020.9316538
  17. Lian, D., Hu, L., Luo, W., Xu, Y., Duan, L., Yu, J., Gao, S. (2019). Multiview Multitask Gaze Estimation With Deep Convolutional Neural Networks. IEEE Transactions on Neural Networks and Learning Systems, 30 (10), 3010–3023. doi: http://doi.org/10.1109/tnnls.2018.2865525
  18. Scott, G. J., Marcum, R. A., Davis, C. H., Nivin, T. W. (2017). Fusion of Deep Convolutional Neural Networks for Land Cover Classification of High-Resolution Imagery. IEEE Geoscience and Remote Sensing Letters, 14 (9), 1638–1642. doi: http://doi.org/10.1109/lgrs.2017.2722988
  19. Li, H., Li, J., Han, X. (2019). Robot Vision Model Based on Multi-Neural Network Fusion. 2019 IEEE 3rd Information Technology Networking Electronic and Automation Control Conference (ITNEC), 2571–2577. doi: http://doi.org/10.1109/itnec.2019.8729210
  20. Knysh, B., Kulyk, Y. (2021). Improving a model of object recognition in images based on a convolutional neural network. Eastern-European Journal of Enterprise Technologies, 3 (9 (111)), 40–50. doi: http://doi.org/10.15587/1729-4061.2021.233786
  21. Krizhevsky, A., Sutskever, I., Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60 (6), 84–90. doi: http://doi.org/10.1145/3065386
  22. Sugoniaev, A. (2018). Iskusstvennye neironnye seti. Funktsiia poter. Saint-Petersburg. Available at: https://ppt-online.org/338631
  23. Slyusar, V. (2021). Neural Networks Models based on the tensor-matrix theory. Problems of the development of promising micro- and nanoelectronic systems (MNS-2021), 23–28. doi: http://doi.org/10.31114/2078-7707-2021-2-23-28
  24. Slyusar, V. I. (1999). A family of face products of matrices and its properties. Cybernetics and Systems Analysis, 35 (3), 379–384. doi: http://doi.org/10.1007/bf02733426
  25. Shchegolev, A. (2020). Development of an element base for superconducting artificial neural networks based on macroscopic quantum effects. Moscow. Available at: https://istina.msu.ru/download/321964642/1kGL1g:Naktuv1mMyFA6kEvKTBPEfux51U/
  26. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research, 15 (56), 1929–1958. Available at: http://jmlr.org/papers/v15/srivastava14a.html

Downloads

Published

2021-10-31

How to Cite

Slyusar, V., Protsenko, M., Chernukha, A., Kovalov, P., Borodych, P., Shevchenko, S., Chernikov, O., Vazhynskyi, S., Bogatov, O., & Khrustalev, K. (2021). Improvement of the model of object recognition in aero photographs using deep convolutional neural networks. Eastern-European Journal of Enterprise Technologies, 5(2 (113), 6–21. https://doi.org/10.15587/1729-4061.2021.243094

Issue

Vol. 5 No. 2 (113) (2021): Information technology. Industry control systems

Section

Information technology

License

Copyright (c) 2021 Vadym Slyusar, Mykhailo Protsenko, Anton Chernukha, Pavlo Kovalov, Pavlo Borodych, Serhii Shevchenko, Oleksandr Chernikov, Serhii Vazhynskyi, Oleg Bogatov, Kirill Khrustalev

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.

A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.