Forecasting software system quality metrics using modifications of the DeepLIFT interpretation method
DOI:
https://doi.org/10.30837/2522-9818.2024.4.110Keywords:
Modeling; DeepLIFT; innovations; quality assessment; neural networks; optimization.Abstract
The subject of the study is to improve the interpretative method of DeepLIFT (Deep Learning Important Features) in the context of ensuring quality assessment of software systems (PS). The algorithmic and mathematical aspects of the DeepLIFT base method, as well as its improvement for analyzing and improving the quality of PS, are being studied. Purpose: development and justification of five methods of optimization modifications of the DeepLIFT method to improve the accuracy and efficiency of forecasting the quality of PS quality. The article solves the following tasks: to conduct a detailed review of problems related to the use of DeepLIFT when assessing the quality of the aircraft; provide a mathematical description of the five ways of modifying the DeepLIFT method aimed at improving the accuracy, adaptability and speed of quality assessment; experimental testing of the proposed modifications is performed to evaluate their effectiveness in assessing the quality of aircraft compared to the original DeepLIFT method. Research methods: analysis of literature; methods of experimental verification; calculation of average values and use of significance tests; modeling. The results were achieved: 1) a detailed analysis of the limitations of the base method of DeepLIFT in the context of modern software systems, which revealed low adaptability to dynamic data, limited interpretability for complex architectures and difficulty with stability of results in cases of variable operating conditions; 2) five DeepLIFT method is proposed; 3) the proposed modifications have undergone experimental testing, which demonstrated their effectiveness compared to the original DeepLIFT method. The results of the study showed that all modifications allow you to achieve improvements in key parameters. Conclusions: in the face of increasing the complexity of architectures, increasing the volume of data and productivity requirements, innovative methods of evaluation and predicting the quality of PS become a necessary. Each of the improved methods increases accuracy and interpretability compared to the base DeepLIFT. The highest accuracy is shown by the temporal DeepLIFT (T-Dlift) respectively.
References
Список літератури
Altmann A., Toloşi L., Sander O., Lengauer T. Permutation importance: A corrected feature importance measure. Bioinformatics, 26 (10), 2010. Р. 1340–1347. DOI: https://doi.org.1093/bioinformatics/btq134
Amoriello T., Ciccoritti R., Ferrante P. Prediction of strawberries’ quality parameters using artificial neural networks. Agronomy, 12 (4), 2022. 963 р. DOI: https://doi.org.3390/agronomy12040963
Antwarg L., Miller R. M., Shapira B., Rokach L. Explaining anomalies detected by autoencoders using Shapley Additive Explanations. Expert Systems With Applications, 186, 2021. 115736 р. DOI: https://doi.org.1016/j.eswa.2021.115736
Dheeraj K. N., Goutham R. J., Arthi L. Crop quality prediction using ml and neural networks. International Journal on Cybernetics & Informatics, 10 (2), 2021. Р. 07–11. DOI: https://doi.org.5121/ijci.2021.100202
EjrnÆs R., Aude E., Nygaard B., Münier B. Prediction of habitat quality using ordination and neural networks. Ecological Applications, 12 (4), 2002. Р. 1180–1187. DOI: 10.1890/1051-0761(2002)012[1180:POHQUO]2.0.CO;2
Fumagalli F., Muschalik M., Hüllermeier E., Hammer B. Incremental permutation feature importance (iPFI): Towards online explanations on data streams. Machine Learning. 2023. Р. 4863–4903. DOI: https://doi.org.1007/s10994-023-06385-y
Ghanmi H., Ghith A., Benameur T. Ring spun yarn quality prediction using hybrid neural networks. The Journal of the Textile Institute, Р. 1–9. 2021. DOI: https://doi.org.1080/00405000.2021.2022826
Jha K. N., Chockalingam C. T. Prediction of quality performance using artificial neural networks. Journal of Advances in Management Research, 6 (1), 2009. Р. 70–86. DOI: https://doi.org.1108/09727980910972172
Kaneko H. Cross‐validated permutation feature importance considering correlation between features. Analytical Science Advances. P. 278-287. 2022. DOI: https://doi.org.1002/ansa.202200018
Michalski A., Duraj K., Kupcewicz B. Leukocyte deep learning classification assessment using Shapley additive explanations algorithm. International Journal of Laboratory Hematology. 2023. P. 297–302. DOI: https://doi.org.1111/ijlh.14031
Miković R., Arsić B., Gligorijević Đ. Importance of social capital for knowledge acquisition-DeepLIFT learning from international development projects. Information Processing & Management, 61 (4), 103694 р. 2024. DOI: https://doi.org.1016/j.ipm.2024.103694
Movsessian A., Cava D. G., Tcherniak D. Interpretable machine learning in damage detection using shapley additive explanations. ASCE-ASME J Risk and Uncert in Engrg Sys Part B Mech Engrg, 8 (2). 2022. 11 р. DOI: https://doi.org/10.1115/1.4053304
Ni H., Yin H. Exchange rate prediction using hybrid neural networks and trading indicators. Neurocomputing, 72 (13-15), 2009. Р. 2815–2823. DOI: https://doi.org.1016/j.neucom.2008.09.023
Nirmalraj S., Antony A. S. M., Srideviponmalar P., Oliver A. S., Velmurugan K. J., Elanangai V., Nagarajan G. Permutation feature importance-based fusion techniques for diabetes prediction. Soft Computing. 2023. DOI: https://doi.org.1007/s00500-023-08041-y
Poligné I., Broyart B., Trystram G., Collignan A. Prediction of mass-transfer kinetics and product quality changes during a dehydration-impregnation-soaking process using artificial neural networks. application to pork curing. LWT - Food Science and Technology, 35(8), 2002. Р. 748–756. DOI: https://doi.org.1006/fstl.2002.0939
Rahman N. H. A., Lee M. H., Suhartono S., Latif M. T. Artificial neural networks and fuzzy time series forecasting: An application to air quality. Quality & Quantity, 49(6), 2014. Р. 2633–2647. DOI: https://doi.org.1007/s11135-014-0132-6
Schweidtmann A. M., Rittig J. G., König A., Grohe M., Mitsos A., Dahmen M. Graph neural networks for prediction of fuel ignition quality. Energy & Fuels, 34 (9), 2020. Р. 11395–11407. DOI: https://doi.org.1021/acs.energyfuels.0c01533
Tan C. H., Yusof K. M., Alwi S. R. W. Quality prediction for polypropylene extrusion based on neural networks. IOP Conference Series: Materials Science and Engineering, 1257 (1), 2022. Р. 1–6. DOI: doi:10.1088/1757-899X/1257/1/012034
Tan J. S. Ablation study on feature group importance for automated essay scoring. Asia-Pacific Journal of Information Technology and Multimedia, 11 (01), 2022. Р. 90–101. DOI: https://doi.org.17576/apjitm-2022-1101-08
Thwin M. M. T., Quah T. S. Application of neural networks for software quality prediction using object-oriented metrics. Journal of Systems and Software, 76 (2), 2005. Р. 147–156. DOI: https://doi.org.1016/j.jss.2004.05.001
Wang S., Zhang Y. Grad-CAM: Understanding AI models. Computers, Materials & Continua, 76 (2), 2023. Р. 1321–1324. DOI: https://doi.org.32604/cmc.2023.041419
References
Altmann, A., Toloşi, L., Sander, O., Lengauer, T. (2010), "Permutation importance: A corrected feature importance measure". Bioinformatics, 26 (10), Р. 1340–1347. DOI: https://doi.org.1093/bioinformatics/btq134
Amoriello, T., Ciccoritti, R., Ferrante, P. (2022), "Prediction of strawberries’ quality parameters using artificial neural networks". Agronomy, 12 (4), 963 р. DOI: https://doi.org.3390/agronomy12040963
Antwarg, L., Miller, R. M., Shapira, B., Rokach, L. (2021), "Explaining anomalies detected by autoencoders using Shapley Additive Explanations". Expert Systems With Applications, 186, 115736 р. DOI: https://doi.org.1016/j.eswa.2021.115736
Dheeraj, K. N., Goutham, R. J., Arthi, L. (2021), "Crop quality prediction using ml and neural networks". International Journal on Cybernetics & Informatics, 10 (2), Р. 07–11. DOI: https://doi.org.5121/ijci.2021.100202
EjrnÆs, R., Aude, E., Nygaard, B., Münier, B. (2002), "Prediction of habitat quality using ordination and neural networks". Ecological Applications, 12 (4), Р. 1180–1187. DOI: 10.1890/1051-0761(2002)012[1180:POHQUO]2.0.CO;2
Fumagalli, F., Muschalik, M., Hüllermeier, E., Hammer, B. (2023), "Incremental permutation feature importance (iPFI): Towards online explanations on data streams". Machine Learning. Р. 4863–4903. DOI: https://doi.org.1007/s10994-023-06385-y
Ghanmi, H., Ghith, A., Benameur, T. (2021), "Ring spun yarn quality prediction using hybrid neural networks". The Journal of the Textile Institute, Р. 1–9. DOI: https://doi.org.1080/00405000.2021.2022826
Jha, K. N., Chockalingam, C. T. (2009), "Prediction of quality performance using artificial neural networks". Journal of Advances in Management Research, 6 (1), Р. 70–86. DOI: https://doi.org.1108/09727980910972172
Kaneko, H. (2022), "Cross‐validated permutation feature importance considering correlation between features". Analytical Science Advances. P. 278-287. DOI: https://doi.org.1002/ansa.202200018
Michalski, A., Duraj, K., Kupcewicz, B. (2023), "Leukocyte deep learning classification assessment using Shapley additive explanations algorithm". International Journal of Laboratory Hematology. P. 297–302. DOI: https://doi.org.1111/ijlh.14031 Miković, R., Arsić, B., Gligorijević, Đ. (2024), "Importance of social capital for knowledge acquisition-DeepLIFT learning from international development projects". Information Processing & Management, 61 (4), 103694 р. DOI: https://doi.org.1016/j.ipm.2024.103694
Movsessian, A., Cava, D. G., Tcherniak, D. (2022), "Interpretable machine learning in damage detection using shapley additive explanations". ASCE-ASME J Risk and Uncert in Engrg Sys Part B Mech Engrg, 8 (2). 11 р. DOI: https://doi.org/10.1115/1.4053304
Ni, H., Yin, H. (2009), "Exchange rate prediction using hybrid neural networks and trading indicators". Neurocomputing, 72 (13–15), Р. 2815–2823. DOI: https://doi.org.1016/j.neucom.2008.09.023
Nirmalraj, S., Antony, A. S. M., Srideviponmalar, P., Oliver, A. S., Velmurugan, K. J., Elanangai, V., Nagarajan, G. (2023), "Permutation feature importance-based fusion techniques for diabetes prediction". Soft Computing. 2023. DOI: https://doi.org.1007/s00500-023-08041-y
Poligné, I., Broyart, B., Trystram, G., Collignan, A. (2002), "Prediction of mass-transfer kinetics and product quality changes during a dehydration-impregnation-soaking process using artificial neural networks. application to pork curing". LWT – Food Science and Technology, 35(8), Р. 748–756. DOI: https://doi.org.1006/fstl.2002.0939
Rahman, N. H. A., Lee, M. H., Suhartono, S., Latif, M. T. (2014), "Artificial neural networks and fuzzy time series forecasting: An application to air quality". Quality & Quantity, 49 (6), Р. 2633–2647. DOI: https://doi.org.1007/s11135-014-0132-6
Schweidtmann, A. M., Rittig, J. G., König, A., Grohe, M., Mitsos, A., (2020), "Dahmen M. Graph neural networks for prediction of fuel ignition quality". Energy & Fuels, 34 (9), Р. 11395-11407. DOI: https://doi.org.1021/acs.energyfuels.0c01533
Tan, C. H., Yusof, K. M., Alwi, S. R. W. (2022), "Quality prediction for polypropylene extrusion based on neural networks". IOP Conference Series: Materials Science and Engineering, 1257 (1), Р. 1–6. DOI: doi:10.1088/1757-899X/1257/1/012034
Tan, J. S. (2022), "Ablation study on feature group importance for automated essay scoring". Asia-Pacific Journal of Information Technology and Multimedia, 11 (01), Р. 90–101. DOI: https://doi.org.17576/apjitm-2022-1101-08
Thwin, M. M. T., Quah, T. S. (2005), "Application of neural networks for software quality prediction using object-oriented metrics". Journal of Systems and Software, 76 (2), 2005. Р. 147–156. DOI: https://doi.org.1016/j.jss.2004.05.001
Wang, S., Zhang, Y. (20230, "Grad-CAM: Understanding AI models". Computers, Materials & Continua, 76 (2), Р. 1321–1324. DOI: https://doi.org.32604/cmc.2023.041419
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Our journal abides by the Creative Commons copyright rights and permissions for open access journals.
Authors who publish with this journal agree to the following terms:
Authors hold the copyright without restrictions and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-commercial and non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
Authors are permitted and encouraged to post their published work online (e.g., in institutional repositories or on their website) as it can lead to productive exchanges, as well as earlier and greater citation of published work.
