Estimating parameters of linear regression with an exponential power distribution of errors by using a polynomial maximization method

Authors

DOI:

https://doi.org/10.15587/1729-4061.2021.225525

Keywords:

regression, exponential power distribution, parameter estimation, moments, polynomial maximization method

Abstract

This paper considers the application of a method for maximizing polynomials in order to find estimates of the parameters of a multifactorial linear regression provided the random errors of the regression model follow an exponential power distribution. The method used is conceptually close to a maximum likelihood method because it is based on the maximization of selective statistics in the neighborhood of the true values of the evaluated parameters. However, in contrast to the classical parametric approach, it employs a partial probabilistic description in the form of a limited number of statistics of higher orders.

The adaptive algorithm of statistical estimation has been synthesized, which takes into consideration the properties of regression residues and makes it possible to find refined values for the estimates of the parameters of a linear multifactorial regression using the numerical Newton-Rafson iterative procedure. Based on the apparatus of the quantity of extracted information, the analytical expressions have been derived that make it possible to analyze the theoretical accuracy (asymptotic variances) of estimates for the method of maximizing polynomials depending on the magnitude of the exponential power distribution parameters.

Statistical modeling was employed to perform a comparative analysis of the variance of estimates obtained using the method of maximizing polynomials with the accuracy of classical methods: the least squares and maximum likelihood. Regions of the greatest efficiency for each studied method have been constructed, depending on the magnitude of the parameter of the form of exponential power distribution and sample size. It has been shown that estimates from the polynomial maximization method may demonstrate a much lower variance compared to the estimates from a least-square method. And, in some cases (for flat-topped distributions and in the absence of a priori information), may exceed the estimates from the maximum likelihood method in terms of accuracy

Author Biographies

Serhii Zabolotnii, Cherkasy State Business College

Doctor of Technical Sciences, Associate Professor

Department of Computer Engineering and Information Technology

Vladyslav Khotunov, Cherkasy State Business College

PhD

Department of Computer Engineering and Information Technology

Anatolii Chepynoha, Cherkasy State Business College

PhD, Associate Professor

Department of Computer Engineering and Information Technology

Olexandr Tkachenko, Cherkasy State Business College

Postgraduate Student

Department of Computer Engineering and Information Technology

References

  1. Subbotin, M. T. (1923). On the law of frequency of error. Mat. Sb., 31 (2), 296–301.
  2. Varanasi, M. K., Aazhang, B. (1989). Parametric generalized Gaussian density estimation. The Journal of the Acoustical Society of America, 86 (4), 1404–1415. doi: https://doi.org/10.1121/1.398700
  3. Nadarajah, S. (2005). A generalized normal distribution. Journal of Applied Statistics, 32 (7), 685–694. doi: https://doi.org/10.1080/02664760500079464
  4. Giller, G. L. (2005). A Generalized Error Distribution. SSRN Electronic Journal. doi: https://doi.org/10.2139/ssrn.2265027
  5. Krasilnikov, A. I. (2019). Family of Subbotin Distributions and its Classification. Electronic modeling, 41 (3), 15–32. doi: https://doi.org/10.15407/emodel.41.03.015
  6. Hassan, M. Y., Hijazi, R. H. (2010). A bimodal exponential power distribution. Pakistan Journal of Statistics, 26 (2), 379–396.
  7. Komunjer, I. (2007). Asymmetric power distribution: Theory and applications to risk measurement. Journal of Applied Econometrics, 22 (5), 891–921. doi: https://doi.org/10.1002/jae.961
  8. Zhu, D., Zinde-Walsh, V. (2009). Properties and estimation of asymmetric exponential power distribution. Journal of Econometrics, 148 (1), 86–99. doi: https://doi.org/10.1016/j.jeconom.2008.09.038
  9. Crowder, G. E., Moore, A. H. (1983). Adaptive Robust Estimation Based on a Family of Generalized Exponential Power Distributions. IEEE Transactions on Reliability, R-32 (5), 488–495. doi: https://doi.org/10.1109/tr.1983.5221739
  10. Mineo, A. M. (2003). On the estimation of the structure parameter of a normal distribution of order p. Statistica, 63(1), 109–122. doi: https://doi.org/10.6092/issn.1973-2201/342
  11. Olosunde, A. A., Soyinka, A. T. (2019). Interval Estimation for Symmetric and Asymmetric Exponential Power Distribution Parameters. Journal of the Iranian Statistical Society, 18 (1), 237–252. doi: https://doi.org/10.29252/jirss.18.1.237
  12. Pogány, T. K., Nadarajah, S. (2010). On the characteristic function of the generalized normal distribution. Comptes Rendus Mathematique, 348 (3-4), 203–206. doi: https://doi.org/10.1016/j.crma.2009.12.010
  13. Novitskiy, P. V., Zograf, I. A. (1991). Otsenka pogreshnostey rezul'tatov izmereniy. Leningrad: Izdatel'stvo Energoatomizdat, 304.
  14. Lindsey, J. K. (1999). Multivariate Elliptically Contoured Distributions for Repeated Measurements. Biometrics, 55 (4), 1277–1280. doi: https://doi.org/10.1111/j.0006-341x.1999.01277.x
  15. Sheluhin, O. I. (1999). Negaussovskie protsessy v radiotehnike. Moscow: Radio i svyaz', 310.
  16. Sharifi, K., Leon-Garcia, A. (1995). Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video. IEEE Transactions on Circuits and Systems for Video Technology, 5 (1), 52–56. doi: https://doi.org/10.1109/76.350779
  17. Dominguez-Molina, J. A., Gonzalez-Farias, G., Rodriguez-Dagnino, R. M. (2001). A practical procedure to estimate the shape parameter in the generalized Gaussian distribution. Available at: http://www.cimat.mx/reportes/enlinea/I-01-18_eng.pdf
  18. Saatci, E., Akan, A. (2010). Respiratory parameter estimation in non-invasive ventilation based on generalized Gaussian noise models. Signal Processing, 90 (2), 480–489. doi: https://doi.org/10.1016/j.sigpro.2009.07.015
  19. Olosunde, A. A. (2013). On exponential power distribution and poultry feeds data: a case study. Journal of The Iranian Statistical Society, 12 (2), 253–269.
  20. Giacalone, M. (2020). A combined method based on kurtosis indexes for estimating p in non-linear Lp-norm regression. Sustainable Futures, 2, 100008. doi: https://doi.org/10.1016/j.sftr.2020.100008
  21. Chan, J. S. K., Choy, S. T. B., Walker, S. G. (2021). On The Estimation Of The Shape Parameter Of A Symmetric Distribution. Journal of Data Science, 18 (1), 78–100. doi: https://doi.org/10.6339/jds.202001_18(1).0004
  22. Olosunde, A. A., Adegoke, A. M. (2016). Goodness-of-fit-test for Exponential Power Distribution. American Journal of Applied Mathematics and Statistics, 4 (1), 1–8.
  23. Giacalone, M., Panarello, D. (2020). Statistical hypothesis testing within the Generalized Error Distribution: Comparing the behavior of some nonparametric techniques. Book of Short Papers SIS 2020, 1338–1343.
  24. Johnson, M. E. (1979). Computer Generation of the Exponential Power Distributions. Journal of Statistical Computation and Simulation, 9, 239–240.
  25. Nardon, M., Pianca, P. (2009). Simulation techniques for generalized Gaussian densities. Journal of Statistical Computation and Simulation, 79 (11), 1317–1329. doi: https://doi.org/10.1080/00949650802290912
  26. Kalke, S., Richter, W.-D. (2013). Simulation of the p-generalized Gaussian distribution. Journal of Statistical Computation and Simulation, 83 (4), 641–667. doi: https://doi.org/10.1080/00949655.2011.631187
  27. Mineo, A. M., Ruggieri, M. (2005). A Software Tool for the Exponential Power Distribution: The normalp Package. Journal of Statistical Software, 12 (4). doi: https://doi.org/10.18637/jss.v012.i04
  28. Rao, C. R. (1945). Information and the accuracy attainable in the estimation of statistical parameters. Bulletin of Calcutta Mathematical Society, 37, 81–89.
  29. Koenker, R., Bassett, G. (1978). Regression Quantiles. Econometrica, 46 (1), 33–50. doi: https://doi.org/10.2307/1913643
  30. Barrodale, I., Roberts, F. D. K. (1973). An Improved Algorithm for Discrete $l_1 $ Linear Approximation. SIAM Journal on Numerical Analysis, 10 (5), 839–848. doi: https://doi.org/10.1137/0710069
  31. Zeckhauser, R., Thompson, M. (1970). Linear Regression with Non-Normal Error Terms. The Review of Economics and Statistics, 52 (3), 280–286. doi: https://doi.org/10.2307/1926296
  32. Mineo, A. (1989). The Norm-P Estimation of Location, Scale and Simple Linear Regression Parameters. Lecture Notes in Statistics, 222–233. doi: https://doi.org/10.1007/978-1-4612-3680-1_26
  33. Agrò, G. (1992). Maximum likelihood and Lp-norm estimators. Statistica Applicata, 4 (2), 171–182.
  34. Tarassenko, P. F., Tarima, S. S., Zhuravlev, A. V., Singh, S. (2014). On sign-based regression quantiles. Journal of Statistical Computation and Simulation, 85 (7), 1420–1441. doi: https://doi.org/10.1080/00949655.2013.875176
  35. Yang, T., Gallagher, C. M., McMahan, C. S. (2018). A robust regression methodology via M-estimation. Communications in Statistics - Theory and Methods, 48 (5), 1092–1107. doi: https://doi.org/10.1080/03610926.2018.1423698
  36. Galea, M., Paula, G. A., Bolfarine, H. (1997). Local influence in elliptical linear regression models. Journal of the Royal Statistical Society: Series D (The Statistician), 46 (1), 71–79. doi: https://doi.org/10.1111/1467-9884.00060
  37. Liu, S. (2002). Local influence in multivariate elliptical linear regression models. Linear Algebra and Its Applications, 354 (1-3), 159–174. doi: https://doi.org/10.1016/s0024-3795(01)00585-7
  38. Ganguly, S. S. (2014). Robust Regression Analysis for Non-Normal Situations under Symmetric Distributions Arising In Medical Research. Journal of Modern Applied Statistical Methods, 13 (1), 446–462. doi: https://doi.org/10.22237/jmasm/1398918480
  39. Bartolucci, F., Scaccia, L. (2005). The use of mixtures for dealing with non-normal regression errors. Computational Statistics & Data Analysis, 48 (4), 821–834. doi: https://doi.org/10.1016/j.csda.2004.04.005
  40. Seo, B., Noh, J., Lee, T., Yoon, Y. J. (2017). Adaptive robust regression with continuous Gaussian scale mixture errors. Journal of the Korean Statistical Society, 46 (1), 113–125. doi: https://doi.org/10.1016/j.jkss.2016.08.002
  41. Tiku, M. L., Islam, M. Q., Selçuk, A. S. (2001). Nonnormal Regression. II. Symmetric Distributions. Communications in Statistics - Theory and Methods, 30 (6), 1021–1045. doi: https://doi.org/10.1081/sta-100104348
  42. Andargie, A. A., Rao, K. S. (2013). Estimation of a linear model with two-parameter symmetric platykurtic distributed errors. Journal of Uncertainty Analysis and Applications, 1 (1). doi: https://doi.org/10.1186/2195-5468-1-13
  43. Stone, C. J. (1975). Adaptive Maximum Likelihood Estimators of a Location Parameter. The Annals of Statistics, 3 (2), 267–284. doi: ttps://doi.org/10.1214/aos/1176343056
  44. Montfort, K., Mooijaart, A., Leeuw, J. (1987). Regression with errors in variables: estimators based on third order moments. Statistica Neerlandica, 41 (4), 223–238. doi: https://doi.org/10.1111/j.1467-9574.1987.tb01215.x
  45. Dagenais, M. G., Dagenais, D. L. (1997). Higher moment estimators for linear regression models with errors in the variables. Journal of Econometrics, 76 (1-2), 193–221. doi: https://doi.org/10.1016/0304-4076(95)01789-5
  46. Cragg, J. G. (1997). Using Higher Moments to Estimate the Simple Errors-in-Variables Model. The RAND Journal of Economics, 28, S71. doi: https://doi.org/10.2307/3087456
  47. Gillard, J. (2014). Method of Moments Estimation in Linear Regression with Errors in both Variables. Communications in Statistics - Theory and Methods, 43 (15), 3208–3222. doi: https://doi.org/10.1080/03610926.2012.698785
  48. Wang, L., Leblanc, A. (2007). Second-order nonlinear least squares estimation. Annals of the Institute of Statistical Mathematics, 60 (4), 883–900. doi: https://doi.org/10.1007/s10463-007-0139-z
  49. Chen, X., Tsao, M., Zhou, J. (2010). Robust second-order least-squares estimator for regression models. Statistical Papers, 53 (2), 371–386. doi: https://doi.org/10.1007/s00362-010-0343-4
  50. Kim, M., Ma, Y. (2011). The efficiency of the second-order nonlinear least squares estimator and its extension. Annals of the Institute of Statistical Mathematics, 64 (4), 751–764. doi: https://doi.org/10.1007/s10463-011-0332-y
  51. Huda, S., Mukerjee, R. (2016). Optimal designs with string property under asymmetric errors and SLS estimation. Statistical Papers, 59 (3), 1255–1268. doi: https://doi.org/10.1007/s00362-016-0819-y
  52. Kunchenko, Yu. P., Lega, Yu. G. (1991). Otsenka parametrov sluchaynyh velichin metodom maksimizatsii polinoma. Kyiv: Naukova dumka, 180.
  53. Zabolotnii, S., Warsza, Z. L., Tkachenko, O. (2018). Polynomial Estimation of Linear Regression Parameters for the Asymmetric PDF of Errors. Advances in Intelligent Systems and Computing, 758–772. doi: https://doi.org/10.1007/978-3-319-77179-3_75
  54. Zabolotnii, S. W., Warsza, Z. L., Tkachenko, O. (2019). Estimation of Linear Regression Parameters of Symmetric Non-Gaussian Errors by Polynomial Maximization Method. Advances in Intelligent Systems and Computing, 636–649. doi: https://doi.org/10.1007/978-3-030-13273-6_59
  55. Warsza, Z. (2017). Ocena niepewności pomiarów o rozkładzie trapezowym metodą maksymalizacji wielomianu. Przemysł Chemiczny, 1 (12), 68–71. doi: https://doi.org/10.15199/62.2017.12.6
  56. Warsza, Z. L., Zabolotnii, S. W. (2017). A Polynomial Estimation of Measurand Parameters for Samples of Non-Gaussian Symmetrically Distributed Data. Advances in Intelligent Systems and Computing, 468–480. doi: https://doi.org/10.1007/978-3-319-54042-9_45
  57. Zabolotnii, S. V., Kucheruk, V. Yu., Warsza, Z. L., Khassenov, А. K. (2018). Polynomial Estimates of Measurand Parameters for Data from Bimodal Mixtures of Exponential Distributions. Vestnik Karagandinskogo universiteta, 2 (90), 71–80.
  58. Warsza, Z. L., Zabolotnii, S. (2018). Estimation of Measurand Parameters for Data from Asymmetric Distributions by Polynomial Maximization Method. Advances in Intelligent Systems and Computing, 746–757. doi: https://doi.org/10.1007/978-3-319-77179-3_74
  59. Zabolotnii, S. W., Warsza, Z. L. (2016). Semi-parametric Estimation of the Change-Point of Parameters of Non-gaussian Sequences by Polynomial Maximization Method. Advances in Intelligent Systems and Computing, 903–919. doi: https://doi.org/10.1007/978-3-319-29357-8_80
  60. Zabolotnii, S. V., Chepynoha, A. V., Bondarenko, Y. Y., Rud, M. P. (2018). Polynomial parameter estimation of exponential power distribution data. Visnyk NTUU KPI Seriia - Radiotekhnika Radioaparatobuduvannia, 75, 40–47, 75, 40–47. doi: https://doi.org/10.20535/radap.2018.75.40-47
  61. Zabolotnii, S. V., Chepynoha, A. V., Chorniy, A. M., Honcharov, A. V. (2020). Сomparative Analysis of Polynomial Maximization and Maximum Likelihood Estimates for Data with Exponential Power Distribution. Visnyk NTUU KPI Seriia - Radiotekhnika Radioaparatobuduvannia, 82, 44–51. doi: https://doi.org/10.20535/radap.2020.82.44-51
  62. Liu, M., Bozdogan, H. (2008). Multivariate regression models with power exponential random errors and subset selection using genetic algorithms with information complexity. European Journal of Pure and Applied Mathematics, 1 (1), 4–37. Available at: https://www.researchgate.net/publication/264947828_Multivariate_regression_models_with_power_exponential_random_errors_and_subset_selection_using_genetic_algorithms_with_information_complexity
  63. Atsedeweyn, A. A., Srinivasa Rao, K. (2013). Linear regression model with new symmetric distributed errors. Journal of Applied Statistics, 41 (2), 364–381. doi: https://doi.org/10.1080/02664763.2013.839638
  64. Ferreira, M. A., Salazar, E. (2014). Bayesian reference analysis for exponential power regression models. Journal of Statistical Distributions and Applications, 1 (1), 12. doi: https://doi.org/10.1186/2195-5832-1-12
  65. Levine, D. M., Krehbiel, T. C., Berenson, M. L. (2000). Business Statistics: A First Course. Prentice-Hall.

Downloads

Published

2021-02-26

How to Cite

Zabolotnii, S., Khotunov, V., Chepynoha, A., & Tkachenko, O. (2021). Estimating parameters of linear regression with an exponential power distribution of errors by using a polynomial maximization method . Eastern-European Journal of Enterprise Technologies, 1(4 (109), 64–73. https://doi.org/10.15587/1729-4061.2021.225525

Issue

Vol. 1 No. 4 (109) (2021): Mathematics and Cybernetics - applied aspects

Section

Mathematics and Cybernetics - applied aspects

License

Copyright (c) 2021 Сергей Васильевич Заболотный, Владислав Игоревич Хотунов, Анатолий Владимирович Чепинога, Александр Николаевич Ткаченко

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.

A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.