THE PLATFORM FOR CREATION OF EVENT-DRIVEN APPLICATIONS BASED ON WOLFRAM MATHEMATICA AND APACHE KAFKA

Authors

DOI:

https://doi.org/10.30837/ITSSI.2021.16.012

Keywords:

event-driven applications, queue manager, mathematical processor, saving resources and funds, cloud technologies, Kafka, Mathematica

Abstract

The article is devoted to the study and development of the mechanism of interaction between Wolfram Mathematica programs and Apache Kafka queue to provide the ability to build event-driven applications based on it. The subject of the research is the practical principles of building a mechanism for interaction between Wolfram Mathematica and Apache Kafka through a proxy-server. The purpose of the article is to develop and substantiate practical recommendations regarding the formation of proxy-server and a mechanism for its work to publishing messages to the Apache Kafka queue and reading messages from it for programs of the mathematical processor Wolfram Mathematica, which will make it possible to build event-driven applications. The tasks are: to determine the mechanism of such interaction, prove the choice of tools for its implementation, create and test the obtained results. The research used the following tools: Apache Kafka, Kafkacat, servers Ubuntu 20 LTS, the method of developing the Wolfram Mathematica package. The results of the research: the mechanism of interaction between Wolfram Mathematica and Apache Kafka through a proxy-server was determined and the corresponding toolkit was created on its basis in the form of two Mathematica packages, which are built on using bash-scripts, Apache Kafka and third-party Kafkacat software. The first - for use on the end user's computer, the second – on a compute server with a remote Mathematica kernel. It is confirmed that the Mathematica processor is currently not suitable in its pure form for real-time data analysis. Conclusions. Practical recommendations have been developed and substantiated regarding the formation of the mechanism of interaction between the Wolfram Mathematica mathematical processor and the Apache Kafka queue manager through a proxy-server for the possibility of working in two directions with the queue: publishing messages and reading them. A toolkit for such interaction in the form of Mathematica packages has been created, their capabilities have been demonstrated. The economic benefit of using the described tools is shown. Future ways of its improvement are given.

Author Biography

Denis Zolotariov

PhD (Physics and Mathematics Sciences)

References

Villamizar, M., Garcés, O., Ochoa, L. et al. (2017), "Cost comparison of running web applications in the cloud using monolithic, microservice, and AWS Lambda architectures", SOCA, Vol. 11, P. 233–247. DOI: https://doi.org/10.1007/s11761-017-0208-y

Gutiérrez–Fernández, A. M., Resinas, M., Ruiz–Cortés, A. (2017), "Redefining a Process Engine as a Microservice Platform", In: Dumas M., Fantinato M. (eds) Business Process Management Workshops. BPM 2016. Lecture Notes in Business Information Processing, Vol. 281, Springer, Cham. DOI: https://doi.org/10.1007/978-3-319-58457-7_19

Brogi, A., Canciani, A., Neri D., Rinaldi, L., Soldani, J. (2018), "Towards a Reference Dataset of Microservice-Based Applications", In: Cerone A., Roveri M. (eds) Software Engineering and Formal Methods. SEFM 2017. Lecture Notes in Computer Science, Vol. 10729, Springer, Cham. DOI: https://doi.org/10.1007/978-3-319-74781-1_16

Monteiro, D., Gadelha, R., Maia, P. H. M., Rocha, L. S., Mendonça, N. C. (2018), "Beethoven: An Event-Driven Lightweight Platform for Microservice Orchestration", In: Cuesta C., Garlan D., Pérez J. (eds) Software Architecture. ECSA 2018. Lecture Notes in Computer Science, Vol. 11048, Springer, Cham. DOI: https://doi.org/10.1007/978-3-030-00761-4_13

GitHub (2020), "Ultimate Message Broker Comparison", available at: https://ultimate-comparisons.github.io/ultimate-message-broker-comparison/ (last accessed 10 December 2020).

G2 (2020), "Best Message Queue (MQ) Software in 2021: Compare Reviews on 40+ MQs", available at: https://www.g2.com/categories/message-queue-mq (last accessed 10 December 2020).

GitHub (2020), "Mathworks-ref-arch/matlab-apache-kafka: MATLAB Interface for Apache Kafka", available at: https://github.com/mathworks-ref-arch/matlab-apache-kafka (last accessed 10 December 2020).

Zolotariov, D. (2021), "The mechanism for creation of event-driven applications based on Wolfram Mathematica and Apache Kafka", Innovative Technologies and Scientific Solutions for Industries, No. 1 (15), P. 53–58. DOI: https://doi.org/10.30837/ITSSI.2021.15.053

Stack Overflow (2020), "Kafka bootstrap-servers vs zookeeper in kafka-console-consumer", available at: https://stackoverflow.com/questions/41774446/kafka-bootstrap-servers-vs-zookeeper-in-kafka-console-consumer (last accessed 10 December 2020).

Stack Overflow (2020), "Apache Kafka - bootstrap-server vs zookeeper params in consumer console", available at: https://stackoverflow.com/questions/53954877/bootstrap-server-vs-zookeeper-params-in-consumer-console (last accessed 10 December 2020).

GitHub (2020), "Edenhill/kafkacat: Generic command line non-JVM Apache Kafka producer and consumer", available at: https://github.com/edenhill/kafkacat (last accessed 10 December 2020).

PuTTY (2020), "Download PuTTY - a free SSH and telnet client for Windows", available at: https://www.putty.org/ (last accessed 10 December 2020).

Wolfram (2020), "Wolfram Mathematica: Modern technical calculations", available at: https://www.wolfram.com/mathematica/ (last accessed 12 November 2020).

Wolfram Library Archive (2020), "Remote Kernel Strategies", available at: https://library.wolfram.com/infocenter/Conferences/7250/ (last accessed 10 December 2020).

Zolotariov, D. (2020), "The distributed system of automated computing based on cloud infrastructure", Innovative Technologies and Scientific Solutions for Industries, No. 4 (14), P. 47–55. DOI: https://doi.org/10.30837/ITSSI.2020.14.047

Zolotariov, D. A. (2020), Automation and optimization of scientific and engineering calculations in Wolfram Mathematica, Kharkiv : FOP Panov A.M., ISBN: 978-617-7859-36-8 [In Ukrainian]

Wolfram Language Documentation (2020), "RemoteConnect", available at: https://reference.wolfram.com/language/ref/RemoteConnect.html (last accessed 10 December 2020).

Downloads

Published

2021-07-06

How to Cite

Zolotariov, D. (2021). THE PLATFORM FOR CREATION OF EVENT-DRIVEN APPLICATIONS BASED ON WOLFRAM MATHEMATICA AND APACHE KAFKA. INNOVATIVE TECHNOLOGIES AND SCIENTIFIC SOLUTIONS FOR INDUSTRIES, (2 (16), 12–18. https://doi.org/10.30837/ITSSI.2021.16.012

Issue

No. 2 (16) (2021): Innovative Technologies and Scientific Solutions for Industries

Section

INFORMATION TECHNOLOGY

License

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Our journal abides by the Creative Commons copyright rights and permissions for open access journals.

Authors who publish with this journal agree to the following terms:

  • Authors hold the copyright without restrictions and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.

  • Authors are able to enter into separate, additional contractual arrangements for the non-commercial and non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.

  • Authors are permitted and encouraged to post their published work online (e.g., in institutional repositories or on their website) as it can lead to productive exchanges, as well as earlier and greater citation of published work.