Development of software and algorithms of parallel learning of artificial neural networks using CUDA technologies

Authors

DOI:

https://doi.org/10.15587/2706-5448.2021.239784

Keywords:

software, artificial neural networks, Python, PyTorch framework, CUDA, modified U-Net architecture

Abstract

The object of research is to parallelize the learning process of artificial neural networks to automate the procedure of medical image analysis using the Python programming language, PyTorch framework and Compute Unified Device Architecture (CUDA) technology. The operation of this framework is based on the Define-by-Run model. The analysis of the available cloud technologies for realization of the task and the analysis of algorithms of learning of artificial neural networks is carried out. A modified U-Net architecture from the MedicalTorch library was used. The purpose of its application was the need for a network that can effectively learn with small data sets, as in the field of medicine one of the most problematic places is the availability of large datasets, due to the requirements for data confidentiality of this nature. The resulting information system is able to implement the tasks set before it, contains the most user-friendly interface and all the necessary tools to simplify and automate the process of visualization and analysis of data. The efficiency of neural network learning with the help of the central processor (CPU) and with the help of the graphic processor (GPU) with the use of CUDA technologies is compared. Cloud technology was used in the study. Google Colab and Microsoft Azure were considered among cloud services. Colab was first used to build a prototype. Therefore, the Azure service was used to effectively teach the finished architecture of the artificial neural network. Measurements were performed using cloud technologies in both services. The Adam optimizer was used to learn the model. CPU duration measurements were also measured to assess the acceleration of CUDA technology. An estimate of the acceleration obtained through the use of GPU computing and cloud technologies was implemented. CPU duration measurements were also measured to assess the acceleration of CUDA technology. The model developed during the research showed satisfactory results according to the metrics of Jaccard and Dyce in solving the problem. A key factor in the success of this study was cloud computing services.

Author Biographies

Yaroslav Sokolovskyy, Lviv Polytechnic National University

Doctor of Technical Sciences, Professor

Department of Computer-Aided Design

Denys Manokhin, Ivan Franko National University of Lviv

Department of Information Systems

Yaroslav Kaplunsky, Ukrainian National Forestry University

Postgraduate Student

Department of Information Technology

Olha Mokrytska, Ukrainian National Forestry University

PhD, Associate Professor

Department of Information Technology

References

  1. Sokolovskyy, Y. I., Shymanskyi, V. M., Mokrytska, O. V., Kharko, Y. V. (2019). Neural network model for identification of material creep curves using CUDA technologies. Ukrainian Journal of Information Technology, 1 (1), 11–16. doi: http://doi.org/10.23939/ujit2019.01.011
  2. Manokhin, D. (2021) Prohramno alhorytmichne zabezpechennia rozparalelennia protsesu navchannia shtuchnykh neironnykh merezh z vykorystanniam tekhnolohii CUDA. Mizhnarodna studentska naukova konferentsiia z pytan prykladnoi matematyky ta kompiuternykh nauk (MSNKPMK-2021). Lviv. Available at: https://ami.lnu.edu.ua/wp-content/uploads/2021/05/Ministerstvo-osvity-i-nauky-Ukrainy.docx
  3. Ambros, R., Waltham, R. et. al. (2021). Godfrey Hounsfield. Available at: https://radiopaedia.org/articles/godfrey-hounsfield?lang=us
  4. Bell, D. J., Mirjan, Pr., Nadrljanski, M. et. al. (2021). Computed tomography. Available at: https://radiopaedia.org/articles/computed-tomography
  5. Bell, D. J., Greenway, K. et. al. (2021). Hounsfield unit. Available at: https://radiopaedia.org/articles/hounsfield-unit
  6. Goodfellow, I., Bengio, Yo., Courville, A. (2016). Deep Learning. MIT Press, 781.
  7. Ciresan, D. C., Gambardella, L. M., Giusti, A. (2012). Deep neural networks segment neuronal membranes in electron microscopy images. NIPS, 2852–2860.
  8. Ronnenbergerm, O., Fischerm, P., Broxm, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer, LNCS, 9351, 234–241. doi: http://doi.org/10.1007/978-3-319-24574-4_28
  9. Chilamkurthy, S., Ghosh, R., Tanamala, S., Biviji, M., Campeau, N. G., Venugopal, V. K., Warier, P. (2018). Development and validation of deep learning algorithms for detection of critical findings in head CT scans. arXiv preprint. Available at: https://arxiv.org/abs/1803.05854
  10. RSNA Intracranial Hemorrhage Detection (2019). Radiological Society of North Ameriaca. Available at: https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/overview
  11. Hssayeni, M. D., Croock, M. S., Salman, A. D., Al-khafaji Hassan Falah, Yahya, Z. A., Ghoraani, B. (2020). Intracranial Hemorrhage Segmentation Using a Deep Convolutional Model. Data, 5 (1), 14. doi: http://doi.org/10.3390/data5010014
  12. Goldberger, A. L., Amaral, L. A., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G. et. al. (2000). PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation, 101 (23), E215–E220. doi: http://doi.org/10.1161/01.cir.101.23.e215
  13. Perone, C. S., Cclauss, Saravia, E., Ballester, P. L., Tare, M. (2018). Perone/medicaltorch: Release v0.2 (v0.2). doi: https://doi.org/10.5281/zenodo.1495335
  14. Hssayeni, M. (2020). Computed Tomography Images for Intracranial Hemorrhage Detection and Segmentation. PhysioNet. 1.3.1. doi: https://doi.org/10.13026/4nae-zg36
  15. Tokui, S., Oono, K. (2015). Chainer: a Next-Generation Open Source Framework for Deep Learning. Available at: http://learningsys.org/papers/LearningSys_2015_paper_33.pdf
  16. Hu, W., Miyato, T., Tokui, S., Matsumoto, E., Sugiyama, M.. (2017). Learning Discrete Representations via Information Maximizing Self-Augmented Training. Proceedings of the 34th International Conference on Machine Learning Proceedings of Machine Learning Research, 70, 1558–1567. Available at: https://proceedings.mlr.press/v70/hu17b.html
  17. PyTorch Documentation (2021) Available at: https://pytorch.org/docs/stable/index.html
  18. Colaboratory Frequently Asked Questions (2021). Available at: https://research.google.com/colaboratory/faq.html
  19. How Azure Machine Learning works: Architecture and concepts (2020). Available at: https://docs.microsoft.com/en-us/azure/machine-learning/concept-azure-machine-learning-architecture

Downloads

Published

2021-09-23

How to Cite

Sokolovskyy, Y., Manokhin, D., Kaplunsky, Y., & Mokrytska, O. (2021). Development of software and algorithms of parallel learning of artificial neural networks using CUDA technologies. Technology Audit and Production Reserves, 5(2(61), 21–25. https://doi.org/10.15587/2706-5448.2021.239784

Issue

Vol. 5 No. 2(61) (2021): Information and control systems

Section

Systems and Control Processes: Original Research

License

Copyright (c) 2021 Yaroslav Sokolovskyy, Denys Manokhin, Yaroslav Kaplunsky, Olha Mokrytska

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.