A theoretically proposed algorithm in a decision tree format for choosing an efficient storage type of large datasets

Authors

DOI:

https://doi.org/10.15587/2706-5448.2022.251281

Keywords:

large datasets, non-relational database, column-oriented database, document-oriented database, key-value database, graph database

Abstract

The object of research is methods and approaches to improve storage efficiency and optimize access to large amounts of data. The importance of this study consists in the wide dissemination of big data and the need for the right selection of technologies that will help improve the efficiency of big data processing systems. The complexity of the choice is caused by the large number of different data storages and databases that are available now, so the best decision requires a deep understanding of the advantages, disadvantages and features of each. And the difficulty lies in the lack of a universal algorithm for deciding on the optimal repository. Accordingly, based on the experiments, analysis of existing projects and research papers, a decision-making algorithm was proposed that determines the best way to store large datasets, depending on their characteristics and additional system requirements. This is necessary to simplify the design of the system in the early stages of big data processing projects. Thus, by highlighting the key differences, as well as the disadvantages and advantages of each type of storage and database, a list of key characteristics of the data and the future system, which should be considered when designing.

This algorithm is a theoretical proposal based on the studied research papers. Accordingly, using this algorithm at the design stage of the system, it would be possible to quickly and clearly determine the optimal type of storage of large datasets. The paper considers column-oriented, document-oriented, graph and key-value types of databases, as well as distributed file systems and cloud services.

Author Biographies

Sofiia Materynska, National Technical University of Ukraine «Igor Sikorsky Kyiv Polytechnic Institute»

Department of System Design

Vadym Yaremenko, National Technical University of Ukraine «Igor Sikorsky Kyiv Polytechnic Institute»

Postgraduent Student, Assistant

Department of System Design

Walery Rogoza, National Technical University of Ukraine «Igor Sikorsky Kyiv Polytechnic Institute»

Doctor of Technical Science, Professor

Department of System Design

References

  1. Dash, S., Shakyawar, S. K., Sharma, M., Kaushik, S. (2019). Big data in healthcare: management, analysis and future prospects. Journal of Big Data, 6 (1). doi: http://doi.org/10.1186/s40537-019-0217-0
  2. Raghupathi, W., Raghupathi, V. (2014). Big data analytics in healthcare: promise and potential. Health Information Science and Systems, 2 (1). doi: http://doi.org/10.1186/2047-2501-2-3
  3. Raja, R., Mukherjee, I., Sarkar, B. K. (2020). A Systematic Review of Healthcare Big Data. Scientific Programming, 2020, 1–15. doi: http://doi.org/10.1155/2020/5471849
  4. Siddiqa, A., Karim, A., Gani, A. (2017). Big data storage technologies: a survey. Frontiers of Information Technology & Electronic Engineering, 18 (8), 1040–1070. doi: http://doi.org/10.1631/fitee.1500441
  5. Kumar, S., Singh, M. (2019). Big data analytics for healthcare industry: impact, applications, and tools. Big Data Mining and Analytics, 2 (1), 48–57. doi: http://doi.org/10.26599/bdma.2018.9020031
  6. Alonso, S. G., de la Torre Díez, I., Rodrigues, J. J. P. C., Hamrioui, S., López-Coronado, M. (2017). A Systematic Review of Techniques and Sources of Big Data in the Healthcare Sector. Journal of Medical Systems, 41 (11). doi: http://doi.org/10.1007/s10916-017-0832-2
  7. Pandey, M. K., Subbiah, K. (2016). A Novel Storage Architecture for Facilitating Efficient Analytics of Health Informatics Big Data in Cloud. 2016 IEEE International Conference on Computer and Information Technology (CIT). doi: http://doi.org/10.1109/cit.2016.86
  8. Olaronke, I., Oluwaseun, O. (2016). Big data in healthcare: Prospects, challenges and resolutions. 2016 Future Technologies Conference (FTC). doi: http://doi.org/10.1109/ftc.2016.7821747
  9. Suthakar, U., Magnoni, L., Smith, D. R., Khan, A., Andreeva, J. (2016). An efficient strategy for the collection and storage of large volumes of data for computation. Journal of Big Data, 3 (1). doi: http://doi.org/10.1186/s40537-016-0056-1
  10. Geihs, M., Buchmann, J.; Lee, K. (Ed.) (2019). ELSA: Efficient Long-Term Secure Storage of Large Datasets. Information Security and Cryptology – ICISC 2018. ICISC 2018. Lecture Notes in Computer Science. Cham: Springer, 269–286. doi: https://doi.org/10.1007/978-3-030-12146-4_17

Downloads

Published

2022-01-19

How to Cite

Materynska, S., Yaremenko, V., & Rogoza, W. (2022). A theoretically proposed algorithm in a decision tree format for choosing an efficient storage type of large datasets. Technology Audit and Production Reserves, 1(2(63), 6–9. https://doi.org/10.15587/2706-5448.2022.251281

Issue

Vol. 1 No. 2(63) (2022): Information and control systems

Section

Information Technologies: Reports on Research Projects

License

Copyright (c) 2022 Sofiia Materynska, Vadym Yaremenko, Walery Rogoza

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.