Design of a framework for serverless distributed data processing using queues

Authors

DOI:

https://doi.org/10.15587/1729-4061.2025.335723

Keywords:

serverless architecture, predictive autoscaling, cloud computing, distributed data processing

Abstract

This study's object is the organization of distributed data processing in cloud environments using serverless computing.

The evolution of serverless computing has led to a change in approaches to building the architecture of applications deployed in the cloud. The ability to abstract from infrastructure management while achieving cost-effectiveness is becoming a significant task requiring new tools. One such tool is a new framework that makes it possible to scale computing resources depending on the workload dynamically, store the state of the computing process using DynamoDB, and provide real-time progress tracking.

A serverless application for the distributed generation of PDF documents has been developed and deployed to test the proposed framework. The real load was emulated using Locust; files containing 1,025,132 records were fed to the application input. The results of the experiments showed that the application started to work 25.8% faster, the throughput increased by 21.3%, and the number of cold starts decreased by 3% compared to conventional scaling.

Additionally, the main areas of further research on developing and improving the designed framework have been identified. This study provides possibilities for predictive automatic scaling using semi-Markov process models in a serverless environment.

Unlike traditional reactive approaches, the proposed approach predicts changes in advance and proactively scales parts of the application, which makes it possible to reduce delays and avoid cold starts. The framework could be used to develop serverless applications for distributed data processing using message queues and the ability to monitor the processing in real time

Author Biographies

Oleksandr Kyrychenko, Yuriy Fedkovych Chernivtsi National University

Assistant 

Department of Mathematical Problems of Control and Cybernetics

Serhii Ostapov, Yuriy Fedkovych Chernivtsi National University

Doctor of Physico-Mathematical Sciences, Professor

Department of Computer Systems Software

Oksana Kyrychenko, Yuriy Fedkovych Chernivtsi National University

PhD, Associate Professor

Department of Mathematical Problems of Control and Cybernetics

References

  1. Thumala, S. (2020). Building Highly Resilient Architectures in the Cloud. Nanotechnology Perceptions, 16 (2), 264–284.
  2. Mampage, A., Karunasekera, S., Buyya, R. (2025). A deep reinforcement learning based algorithm for time and cost optimized scaling of serverless applications. Future Generation Computer Systems, 173, 107873. https://doi.org/10.1016/j.future.2025.107873
  3. Sohani, M., Jain, S. C. (2021). A Predictive Priority-Based Dynamic Resource Provisioning Scheme With Load Balancing in Heterogeneous Cloud Computing. IEEE Access, 9, 62653–62664. https://doi.org/10.1109/access.2021.3074833
  4. Beikzadeh Abbasi, F., Rezaee, A., Adabi, S., Movaghar, A. (2023). Fault-tolerant scheduling of graph-based loads on fog/cloud environments with multi-level queues and LSTM-based workload prediction. Computer Networks, 235, 109964. https://doi.org/10.1016/j.comnet.2023.109964
  5. Ghorbian, M., Ghobaei-Arani, M. (2024). A survey on the cold start latency approaches in serverless computing: an optimization-based perspective. Computing, 106 (11), 3755–3809. https://doi.org/10.1007/s00607-024-01335-5
  6. Tari, M., Ghobaei-Arani, M., Pouramini, J., Ghorbian, M. (2024). Auto-scaling mechanisms in serverless computing: A comprehensive review. Computer Science Review, 53, 100650. https://doi.org/10.1016/j.cosrev.2024.100650
  7. Manchana, R. (2020). Operationalizing Batch Workloads in the Cloud with Case Studies. International Journal of Science and Research (IJSR), 9 (7), 2031–2041. https://doi.org/10.21275/sr24820052154
  8. Alharthi, S., Alshamsi, A., Alseiari, A., Alwarafy, A. (2024). Auto-Scaling Techniques in Cloud Computing: Issues and Research Directions. Sensors, 24 (17), 5551. https://doi.org/10.3390/s24175551
  9. Taha, M. B., Sanjalawe, Y., Al-Daraiseh, A., Fraihat, S., Al-E’mari, S. R. (2024). Proactive Auto-Scaling for Service Function Chains in Cloud Computing Based on Deep Learning. IEEE Access, 12, 38575–38593. https://doi.org/10.1109/access.2024.3375772
  10. Verma, S., Bala, A. (2021). Auto-scaling techniques for IoT-based cloud applications: a review. Cluster Computing, 24 (3), 2425–2459. https://doi.org/10.1007/s10586-021-03265-9
  11. Kyrychenko, O. (2024). Real-time communication tools for web applications in a cloud environment. The 13th International Conference on Electronics, Communications and Computing's (IC ECCO), 127–128. Available at: https://ecco.utm.md/wp-content/uploads/2024/12/IC-ECCO-2024-AbstractBookBN.pdf
  12. Nastic, S., Rausch, T., Scekic, O., Dustdar, S., Gusev, M., Koteska, B. et al. (2017). A Serverless Real-Time Data Analytics Platform for Edge Computing. IEEE Internet Computing, 21 (4), 64–71. https://doi.org/10.1109/mic.2017.2911430
  13. Kaur, N., Mittal, A. (2021). Fog Computing Serverless Architecture for Real Time Unpredictable Traffic. IOP Conference Series: Materials Science and Engineering, 1022 (1), 012026. https://doi.org/10.1088/1757-899x/1022/1/012026
  14. Amazon Simple Storage Service Documentation. Amazon Web Services. Available at: https://docs.aws.amazon.com/s3
  15. Amazon Simple Queue Service. Amazon Web Services. Available at: https://aws.amazon.com/sqs
  16. Amazon Web Services. (2025). What is AWS Lambda? Available at: https://docs.aws.amazon.com/lambda/latest/dg/welcome.html
  17. Amazon Web Services. (2025). Amazon DynamoDB. Available at: https://aws.amazon.com/dynamodb/.
  18. What is AWS AppSync? Amazon Web Services. Available at: https://docs.aws.amazon.com/appsync/latest/devguide/what-is-appsync.html
  19. Amazon SageMaker. Amazon Web Services. Available at: https://aws.amazon.com/sagemaker/
  20. Amazon EventBridge. Amazon Web Services. Available at: https://aws.amazon.com/eventbridge/
  21. Rojas, L., Yepes, V., Garcia, J. (2025). Complex Dynamics and Intelligent Control: Advances, Challenges, and Applications in Mining and Industrial Processes. Mathematics, 13 (6), 961. https://doi.org/10.3390/math13060961
  22. Kyrychenkо, O. O., Ostapov, S. E., Kyrychenko, O. L. (2025). Optimization of SQS Configurations for Efficient Batch Data Processing. WSEAS Transactions On Systems, 24, 36–43. Portico. https://doi.org/10.37394/23202.2025.24.4
  23. Golec, M., Walia, G. K., Kumar, M., Cuadrado, F., Gill, S. S., Uhlig, S. (2024). Cold Start Latency in Serverless Computing: A Systematic Review, Taxonomy, and Future Directions. ACM Computing Surveys, 57 (3), 1–36. https://doi.org/10.1145/3700875
  24. Saravana Kumar, N., Selvakumara Samy, S. (2025). Cold Start Prediction and Provisioning Optimization in Serverless Computing Using Deep Learning. Concurrency and Computation: Practice and Experience, 37 (4-5). https://doi.org/10.1002/cpe.8392
  25. Nguyen, T. N. (2024). Holistic cold-start management in serverless computing cloud with deep learning for time series. Future Generation Computer Systems, 153, 312–325. https://doi.org/10.1016/j.future.2023.12.011
Design of a framework for serverless distributed data processing using queues

Downloads

Published

2025-08-29

How to Cite

Kyrychenko, O., Ostapov, S., & Kyrychenko, O. (2025). Design of a framework for serverless distributed data processing using queues. Eastern-European Journal of Enterprise Technologies, 4(9 (136), 19–25. https://doi.org/10.15587/1729-4061.2025.335723

Issue

Vol. 4 No. 9 (136) (2025): Information and controlling system

Section

Information and controlling system

License

Copyright (c) 2025 Oleksandr Kyrychenko, Serhii Ostapov, Оksana Kyrychenko

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.

A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.