Design and implementation of the distributed system using an orchestrator based on the data flow paradigm

Authors

DOI:

https://doi.org/10.15587/2706-5448.2020.205151

Keywords:

data flow control paradigm, distributed systems, high-power computing, Internet of things devices, microservice choreography

Abstract

The object of this research is distributed systems under the control of an orchestrator based on the data flow control paradigm, as well as microservice management methods. One of the most problematic places of modern distributed systems is the choice of a method for controlling the logic of the microservice and the processes of interaction between them. The existing concepts of microservice orchestration and choreography do not allow to fully use and distribute the load evenly throughout the system, which is primarily due to the heterogeneous nature of the distributed environment.

As part of the research, the concept of hybrid orchestration based on the paradigm of data flow control is proposed. This approach allows the orchestrator to be used only to initiate a «wave» of calculations on the microservice tree, and the microservices themselves are responsible for the further calculation and dissemination of data. This approach, unlike others, combines the more optimal qualities of orchestration: simple and understandable, at each stage of calculation, system management, coordinated microservice actions. Also, the use of a specialized hybrid orchestrator eliminated one of the main drawbacks, namely, it reduced the responsibility and the amount of computational burden assigned to the distributed system orchestrator, and computation nodes. As a result of an experiment using a distributed system with an orchestra, based on the data flow control paradigm, a several-fold decrease in the load on one orchestra was achieved. This makes it possible to use microcontrollers such as ESP8266, ESP32, Raspberry Pi as a distributed system. Such microcontrollers can act not only as an orchestrator, but also as dataflow nodes. At the same time, the data flow control paradigm allows to evenly and efficiently distribute the load on the system due to the fact that the input data of the system are presented in the form of a computational graph, where each node is a separate microservice.

Author Biographies

Yevhenii Kubiuk, National Technical University of Ukraine «Igor Sikorsky Kyiv Polytechnic Institute», 37, Peremohy ave., Kyiv, Ukraine, 03056

Department of System Design

Kostiantyn Kharchenko, National Technical University of Ukraine «Igor Sikorsky Kyiv Polytechnic Institute», 37, Peremohy ave., Kyiv, Ukraine, 03056

PhD

Department of System Design

References

  1. Dragoni, N., Giallorenzo, S., Lafuente, A. L., Mazzara, M., Montesi, F., Mustafin, R., Safina, L. (2017). Microservices: yesterday, today, and tomorrow. Present and ulterior software engineering. Cham: Springer, 195–216. doi: http://doi.org/10.1007/978-3-319-67425-4_12
  2. Petrenko, A. I., Bulakh, B. V. (2018). Intelligent Service Discovery and Orchestration. Proc. of 2018 IEEE First International Conference on System Analysis and Intelligent Computing (SAIC). Kyiv, 201–205. doi: http://doi.org/10.1109/saic.2018.8516723
  3. Petrenko, A., Bulakh, B. (2019). Automatic Service Orchestration for e-Health Application. Advances in Science, Technology and Engineering Systems Journal, 4 (4), 244–250. doi: http://doi.org/10.25046/aj040430
  4. Petrenko, O. O. (2015). Porivniannia typiv arkhitektury system servisiv. Systemni doslidzhennia i informatsiini tekhnolohii, 4, 48–62.
  5. Kharchenko, K. V. (2016). An Architecture and Test Implementation of Data Flow Virtual Machine. System Analysis and Informatin Technology Conference. Kyiv: IASA NTUU-KPI, 268.
  6. Kharchenko, K., Beznosyk, O., Romanov, V. (2018). Implementation of Neural Networks with Help of a Data Flow Virtual Machine. 2018 IEEE Second International Conference on Data Stream Mining & Processing (DSMP), 407–409. doi: http://doi.org/10.1109/dsmp.2018.8478455
  7. Kharchenko, K., Beznosyk, O., Romanov, V. (2017). A set of instructions for data flow virtual machine. 2017 IEEE First Ukraine Conference on Electrical and Computer Engineering (UKRCON). Kyiv, 931–934. doi: http://doi.org/10.1109/ukrcon.2017.8100385
  8. Netflix Conductor. Available at:https://netflix.github.io/conductor/
  9. Zeebe. Available at:https://docs.zeebe.io/index.html
  10. Uber Cadence. Available at:https://cadenceworkflow.io/

Downloads

Published

2020-06-30

How to Cite

Kubiuk, Y., & Kharchenko, K. (2020). Design and implementation of the distributed system using an orchestrator based on the data flow paradigm. Technology Audit and Production Reserves, 3(2(53), 38–41. https://doi.org/10.15587/2706-5448.2020.205151

Issue

Vol. 3 No. 2(53) (2020): Information and control systems

Section

Reports on research projects

License

Copyright (c) 2020 Yevhenii Kubiuk, Kostiantyn Kharchenko

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.