Developing a multi-step recurrent algorithm to maximize the criteria of correntropy

Authors

DOI:

https://doi.org/10.15587/1729-4061.2021.225765

Keywords:

correntropy, multi-step algorithm, kernel width, information weighting factor, algorithm memory, permanence

Abstract

This paper considers the task of constructing a linear model of the object studied using a robust criterion. The functionality applied, in this case, is correntropy. That makes it possible to obtain estimates that have robust properties. The evaluation algorithm is a multi-step procedure that employs a limited number of information measurements, that is, it has limited memory. The feature of the algorithm is that the matrices and observation vectors involved in estimate construction are formed in the following way: they include information about the newly arrived measurements and exclude information about the oldest ones. Depending on the way these matrices and vectors are built (new information is added first, and then outdated is excluded, or the outdated is first excluded, and then a new one is added), two estimate forms are possible. The second Lyapunov method is used to study the convergence of the algorithm. The conditions of convergence for a multi-step algorithm have been defined. The analysis of the established regime has revealed that the algorithm ensures that unbiased estimates are obtained.

It should be noted that all the estimates reported in this work depend on the choice of the width of the nucleus, the information weighting factor, and the algorithm memory, the task of determining which remains open. Therefore, these parameters' estimates should be applied for the practical use of such multi-step algorithms.

The estimates obtained in this paper allow the researcher to pre-evaluate the possibilities of identification using a multi-step algorithm, as well as the effectiveness of its application when solving practical tasks

Author Biographies

Oleg Rudenko, Kharkiv National University of Radio Electronics

Doctor of Technical Sciences, Professor, Head of Department

Department of Сomputer Intelligent Technologies and Systems

Oleksandr Bezsonov, Kharkiv National University of Radio Electronics

Doctor of Technical Sciences, Professor

Department of Сomputer Intelligent Technologies and Systems

Victor Borysenko, Kharkiv National University of Radio Electronics

PhD, Associate Professor

Department of Electronic Computers

Tetiana Borysenko, Kharkiv National University of Radio Electronics

PhD, Associate Professor

Department of Electronic Computers

Sergii Lyashenko, Kharkiv Petro Vasylenko National Technical University of Agriculture

Doctor of Technical Sciences, Professor

Department of Life Safety and Law

References

  1. Tsypkin, Ya. Z., Polyak, B. T. (1977) Ogrublenniy metod maksimal'nogo pravdopodobiya. V kn. Dinamika sistem. Gor'kiy, 12, 22–46.
  2. Tiange Shao, Zheng, Y. R., Benesty, J. (2010). An Affine Projection Sign Algorithm Robust Against Impulsive Interferences. IEEE Signal Processing Letters, 17 (4), 327–330. doi: https://doi.org/10.1109/lsp.2010.2040203
  3. Shin, J., Yoo, J., Park, P. (2012). Variable step-size affine projection sign algorithm. Electronics Letters, 48 (9), 483. doi: https://doi.org/10.1049/el.2012.0751
  4. Lu, L., Zhao, H., Li, K., Chen, B. (2015). A Novel Normalized Sign Algorithm for System Identification Under Impulsive Noise Interference. Circuits, Systems, and Signal Processing, 35 (9), 3244–3265. doi: https://doi.org/10.1007/s00034-015-0195-1
  5. Huang, H.-C., Lee, J. (2012). A New Variable Step-Size NLMS Algorithm and Its Performance Analysis. IEEE Transactions on Signal Processing, 60 (4), 2055–2060. doi: https://doi.org/10.1109/tsp.2011.2181505
  6. Casco-Sánchez, F. M., Medina-Ramírez, R. C., López-Guerrero, M. (2011). A New Variable Step-Size NLMS Algorithm and its Performance Evaluation in Echo Cancelling Applications. Journal of Applied Research and Technology, 9 (03). doi: https://doi.org/10.22201/icat.16656423.2011.9.03.425
  7. Huber, P. J. (1977). Robust methods of estimation of regression coefficients. Series Statistics, 8 (1), 41–53. doi: https://doi.org/10.1080/02331887708801356
  8. Hampel, F. R. (1974). The Influence Curve and its Role in Robust Estimation. Journal of the American Statistical Association, 69 (346), 383–393. doi: https://doi.org/10.1080/01621459.1974.10482962
  9. Adamczyk, T. (2017). Application of the Huber and Hampel M-estimation in real estate value modeling. Geomatics and Environmental Engineering, 11 (1), 15. doi: https://doi.org/10.7494/geom.2017.11.1.15
  10. Rudenko, O. G., Bezsonov, O. O. (2011). Robust training of radial basis networks. Cybernetics and Systems Analysis, 47 (6), 38–46.
  11. Rudenko, O. G., Bezsonov, O. O. (2014). Robust Neuroevolutionary Identification of Nonlinear Nonstationary Objects. Cybernetics and Systems Analysis, 50 (1), 17–30. doi: https://doi.org/10.1007/s10559-014-9589-5
  12. Rudenko, O. G., Bezsonov, O. O., Rudenko, S. О. (2013). Robastnaya identifikatsiya nelineynyh obektov s pomosch'yu evolyutsioniruyuschey radial'no-bazisnoy seti. Cybernetics and Systems Analysis, 49 (2), 15–26.
  13. Rudenko, O., Bezsonov, O. (2011). Function Approximation Using Robust Radial Basis Function Networks. Journal of Intelligent Learning Systems and Applications, 03 (01), 17–25. doi: https://doi.org/10.4236/jilsa.2011.31003
  14. Chambers, J. A., Tanrikulu, O., Constantinides, A. G. (1994). Least mean mixed-norm adaptive filtering. Electronics Letters, 30 (19), 1574–1575. doi: https://doi.org/10.1049/el:19941060
  15. Rakesh, P., Kumar, T. K., Albu, F. (2019). Modified Least-Mean Mixed-Norm Algorithms For Adaptive Sparse System Identification Under Impulsive Noise Environment. 2019 42nd International Conference on Telecommunications and Signal Processing (TSP). doi: https://doi.org/10.1109/tsp.2019.8768813
  16. Papoulis, E. V., Stathaki, T. (2004). A Normalized Robust Mixed-Norm Adaptive Algorithm for System Identification. IEEE Signal Processing Letters, 11 (1), 56–59. doi: https://doi.org/10.1109/lsp.2003.819353
  17. Rudenko, O., Bezsonov, O., Lebediev, O., Serdiuk, N. (2019). Robust identification of non-stationary objects with nongaussian interference. Eastern-European Journal of Enterprise Technologies, 5 (4 (101)), 44–52. doi: https://doi.org/10.15587/1729-4061.2019.181256
  18. Walach, E., Widrow, B. (1984). The least mean fourth (LMF) adaptive algorithm and its family. IEEE Transactions on Information Theory, 30 (2), 275–283. doi: https://doi.org/10.1109/tit.1984.1056886
  19. Zhang, S., Zhang, J. (2015). Fast stable normalised least‐mean fourth algorithm. Electronics Letters, 51 (16), 1276–1277. doi: https://doi.org/10.1049/el.2015.0421
  20. Guan, S., Meng, C., Biswal, B. (2019). Optimal step-size of least mean absolute fourth algorithm in low SNR. arXiv.org. Available at: https://arxiv.org/ftp/arxiv/papers/1908/1908.08165.pdf
  21. Asad, S. M., Moinuddin, M., Zerguine, A., Chambers, J. (2019). A robust and stable variable step-size design for the least-mean fourth algorithm using quotient form. Signal Processing, 162, 196–210. doi: https://doi.org/10.1016/j.sigpro.2019.04.021
  22. Bin Mansoor, U., Mayyala, Q., Moinuddin, M., Zerguine, A. (2017). Quasi-Newton least-mean fourth adaptive algorithm. 2017 25th European Signal Processing Conference (EUSIPCO). doi: https://doi.org/10.23919/eusipco.2017.8081689
  23. Sadiq, A., Usman, M., Khan, S., Naseem, I., Moinuddin, M., Al-Saggaf, U. M. (2019). q-LMF: Quantum Calculus-Based Least Mean Fourth Algorithm. Fourth International Congress on Information and Communication Technology, 303–311. doi: https://doi.org/10.1007/978-981-15-0637-6_25
  24. Zerguine, A., Cowan, C. F. N., Bettayeb, M. (1996). LMS-LMF adaptive scheme for echo cancellation. Electronics Letters, 32 (19), 1776. doi: https://doi.org/10.1049/el:19961202
  25. Zerguine, A., Aboulnasr, T. (2000). Convergence analysis of the variable weight mixed-norm LMS-LMF adaptive algorithm. Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers (Cat. No.00CH37154). doi: https://doi.org/10.1109/acssc.2000.910959
  26. Zerguine, A. (2012). A variable-parameter normalized mixed-norm (VPNMN) adaptive algorithm. EURASIP Journal on Advances in Signal Processing, 2012 (1). doi: https://doi.org/10.1186/1687-6180-2012-55
  27. Rudenko, O., Bezsonov, O., Serdiuk, N., Oliinyk, K., Romanyk, O. (2020). Workable identification of objects based on minimization of combined functional. Information Processing Systems, 1 (160), 80–88. doi: https://doi.org/10.30748/soi.2020.160.10
  28. Rudenko, O., Bezsonov, O., Lebediev, O., Lebediev, V., Oliinyk, K. (2020). Studying the properties of a robust algorithm for identifying linear objects, which minimizes a combined functional. Eastern-European Journal of Enterprise Technologies, 4 (4 (106)), 37–46. doi: https://doi.org/10.15587/1729-4061.2020.210129
  29. Tanrikulu, O., Constantinides, A. G. (1994). Least-mean kurtosis: A novel higher-order statistics based adaptive filtering algorithm. Electronics Letters, 30 (3), 189–190. doi: https://doi.org/10.1049/el:19940129
  30. Pazaitis, D. I., Constantinides, A. G. (1999). A novel kurtosis driven variable step-size adaptive algorithm. IEEE Transactions on Signal Processing, 47 (3), 864–872. doi: https://doi.org/10.1109/78.747793
  31. Engel, Y., Mannor, S., Meir, R. (2004). The Kernel Recursive Least-Squares Algorithm. IEEE Transactions on Signal Processing, 52 (8), 2275–2285. doi: https://doi.org/10.1109/tsp.2004.830985
  32. Gil-Cacho, J. M., Signoretto, M., van Waterschoot, T., Moonen, M., Jensen, S. H. (2013). Nonlinear Acoustic Echo Cancellation Based on a Sliding-Window Leaky Kernel Affine Projection Algorithm. IEEE Transactions on Audio, Speech, and Language Processing, 21 (9), 1867–1878. doi: https://doi.org/10.1109/tasl.2013.2260742
  33. Eksioglu, E. M., Tanc, A. K. (2011). RLS Algorithm With Convex Regularization. IEEE Signal Processing Letters, 18 (8), 470–473. doi: https://doi.org/10.1109/lsp.2011.2159373
  34. Principe, J. C., Xu, D., Zhao, Q., Fisher, J. W. (2000). Learning from examples with information theoretic criteria. Journal of VLSI signal processing systems for signal, image and video technology, 26, 61–77. doi: https://doi.org/10.1023/A:1008143417156
  35. Principe, J. C., Xu, D., Fisher, J. (2000). Information-theoretic learning. Unsupervised Adaptive Filtering. Wiley, 265–319.
  36. Santamaria, I., Pokharel, P. P., Principe, J. C. (2006). Generalized correlation function: definition, properties, and application to blind equalization. IEEE Transactions on Signal Processing, 54 (6), 2187–2197. doi: https://doi.org/10.1109/tsp.2006.872524
  37. Liu, W., Pokharel, P. P., Principe, J. C. (2007). Correntropy: Properties and Applications in Non-Gaussian Signal Processing. IEEE Transactions on Signal Processing, 55 (11), 5286–5298. doi: https://doi.org/10.1109/tsp.2007.896065
  38. Wang, W., Zhao, J., Qu, H., Chen, B., Principe, J. C. (2015). An adaptive kernel width update method of correntropy for channel estimation. 2015 IEEE International Conference on Digital Signal Processing (DSP). doi: https://doi.org/10.1109/icdsp.2015.7252010
  39. Ma, W., Qu, H., Gui, G., Xu, L., Zhao, J., Chen, B. (2015). Maximum correntropy criterion based sparse adaptive filtering algorithms for robust channel estimation under non-Gaussian environments. Journal of the Franklin Institute, 352 (7), 2708–2727. doi: https://doi.org/10.1016/j.jfranklin.2015.03.039
  40. Guo, Y., Ma, B., Li, Y. (2020). A Kernel-Width Adaption Diffusion Maximum Correntropy Algorithm. IEEE Access, 8, 33574–33587. doi: https://doi.org/10.1109/access.2020.2972905
  41. Shi, L., Zhao, H., Zakharov, Y. (2020). An Improved Variable Kernel Width for Maximum Correntropy Criterion Algorithm. IEEE Transactions on Circuits and Systems II: Express Briefs, 67 (7), 1339–1343. doi: https://doi.org/10.1109/tcsii.2018.2880564
  42. Huang, F., Zhang, J., Zhang, S. (2017). Adaptive Filtering Under a Variable Kernel Width Maximum Correntropy Criterion. IEEE Transactions on Circuits and Systems II: Express Briefs, 64 (10), 1247–1251. doi: https://doi.org/10.1109/tcsii.2017.2671339
  43. Lu, L., Zhao, H. (2017). Active impulsive noise control using maximum correntropy with adaptive kernel size. Mechanical Systems and Signal Processing, 87, 180–191. doi: https://doi.org/10.1016/j.ymssp.2016.10.020
  44. Rudenko, O. G., Bezsonov, O. O. (2020). ADALINE Robust Multistep Training Algorithm. Control Systems and Computers, 3 (287), 15–27. doi: https://doi.org/10.15407/csc.2020.03.015

Downloads

Published

2021-02-26

How to Cite

Rudenko, O., Bezsonov, O. ., Borysenko, V. ., Borysenko, T. ., & Lyashenko, S. . (2021). Developing a multi-step recurrent algorithm to maximize the criteria of correntropy. Eastern-European Journal of Enterprise Technologies, 1(4 (109), 54–63. https://doi.org/10.15587/1729-4061.2021.225765

Issue

Vol. 1 No. 4 (109) (2021): Mathematics and Cybernetics - applied aspects

Section

Mathematics and Cybernetics - applied aspects

License

Copyright (c) 2021 Олег Григорьевич Руденко , Александр Александрович Бессонов , Виктор Петрович Борисенко , Татьяна Ивановна Борисенко , Сергей Алексеевич Ляшенко

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.

A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.