Methodology of probabilistic analysis of state dynamics of multi­dimensional semi­Markov dynamic systems

Yelyzaveta Meleshko, Lev Raskin, Serhii Semenov, Oksana Sira


The problem of probabilistic analysis of a complex dynamic system, which in the process of functioning passes from one state to another at random times, is considered. The methodology for calculating the conditional probabilities of the system getting into a given state at a given time t, provided that at the initial time the system was in any of the possible states is proposed. The initial data for analysis are a set of experimentally obtained values of the duration of the system stay in each of the states before transition to another state. Approximation of the resulting histograms using the Erlang distribution gives a set of distribution densities of the duration of the system stay in possible states before transition to other states. At the same time, the choice of the proper Erlang distribution order provides an adequate description of the semi-Markov processes occurring in the system. The mathematical model that relates the obtained distribution densities to the functions determining the probabilistic dynamics of the system is proposed. The model describes a random process of system transitions from any possible initial state to any other state during a given time interval. Using the model, a system of integral equations for the desired functions describing the probabilistic transition process is obtained. To solve these equations, the Laplace transform is used. As a result of solving the system of integral equations, functions are obtained that specify the probability distribution of the system states at any time t. The same functions also describe the asymptotic probability distribution of states. An illustrative example of solving the problem for the case when the distribution densities of the lengths of the system stay in possible states are described by the second-order Erlang distributions is given. The solution procedure is described in detail for the most natural special case, when the initial state is H0


dynamic system with many possible states; random transition process; integral dynamic equations; Laplace transforms


Berzh, K. (1962). Teoriya grafov i ee prilozheniya. Moscow: IL, 320.

Distel', R. (2002). Teoriya grafov. Novosibirsk: IM, 336.

Tihonov, V. I., Mironov, M. A. (1977). Markovskie protsessy. Moscow: Sovetskoe Radio, 481.

Bulinskiy, A. N., SHiryaev, A. N. (2005). Teoriya sluchaynyh protsessov. Moscow: Fizmatgiz, 364.

Kemeni, Dzh., Snell, Dzh. (1970). Konechnye tsepi Markova. Moscow: Nauka, 198.

Chzhun, K.-L. (1954). Odnorodnye tsepi Markova. Moscow: Mir, 264.

Barucha, R. A. (1969). Elementy teorii Markovskih protsessov. Moscow: Nauka, 320.

Dynkin, E. B. (1963). Markovskie protsessy. Moscow: Fizmatgiz, 482.

Cao, X.-R. (2015). Optimization of Average Rewards of Time Nonhomogeneous Markov Chains. IEEE Transactions on Automatic Control, 60 (7), 1841–1856. doi:

Dimitrakos, T. D., Kyriakidis, E. G. (2008). A semi-Markov decision algorithm for the maintenance of a production system with buffer capacity and continuous repair times. International Journal of Production Economics, 111 (2), 752–762. doi:

Feinberg, E. A., Yang, F. (2015). Optimal pricing for a GI/M/k/N queue with several customer types and holding costs. Queueing Systems, 82 (1-2), 103–120. doi:

Li, Q.-L. (2016). Nonlinear Markov processes in big networks. Special Matrices, 4 (1). doi:

Li, Q.-L., Lui, J. C. S. (2014). Block-structured supermarket models. Discrete Event Dynamic Systems, 26 (2), 147–182. doi: 10.

Okamura, H., Miyata, S., Dohi, T. (2015). A Markov Decision Process Approach to Dynamic Power Management in a Cluster System. IEEE Access, 3, 3039–3047. doi:

Sanajian, N., Abouee-Mehrizi, H., Balcıog̃lu, B. (2010). Scheduling policies in the M/G/1 make-to-stock queue. Journal of the Operational Research Society, 61 (1), 115–123. doi:

Krasnov, M. L. (1985). Integral'nye uravneniya. Moscow: Nauka, 476.

Il'in, V. A. (1965). Osnovy matematicheskogo analiza. Moscow: Nauka, 572.

Sveshnikov, A. G., Tihonov, A. N. (1967). Teoriya funktsiy kompleksnoy peremennoy. Moscow: Nauka, 308.

GOST Style Citations

Copyright (c) 2019 Yelyzaveta Meleshko, Lev Raskin, Serhii Semenov, Oksana Sira

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

ISSN (print) 1729-3774, ISSN (on-line) 1729-4061