Use of generative artificial intelligence to improve output message effectiveness in decision support systems for prosumers

Authors

DOI:

https://doi.org/10.15587/2706-5448.2025.333726

Keywords:

generative artificial intelligence, decision support system, prosumers, user experience, photovoltaics

Abstract

The object of this study is the use of generative artificial intelligence (GenAI) to create output messages in a decision support system (DSS) for prosumers. The research addresses the challenge of improving user experience (UX) by enhancing the effectiveness of DSS messages. A prototype DSS was developed for a specific private household equipped with solar panels. A rule-based message generation system was created as a baseline for comparison. An evaluation was conducted through surveys in Ukrainian and English. GenAI models from OpenAI and Anthropic were compared. Messages were assessed along two key dimensions of UX quality: usefulness and ease of comprehension.

The results indicate that GenAI can enhance the effectiveness of DSS recommendations for specific user groups without adverse effects. The Sonnet 3.5 model (Anthropic) generated messages that were rated as statistically more useful (p < 0.05) by female users in Ukrainian. Users preferred shorter messages in English, and Sonnet 3.5 outperformed GPT-4 (OpenAI) in terms of usefulness in both languages (p < 0.05).

The higher usefulness ratings can be attributed to more detailed recommendations while maintaining natural language. The English-language results were likely influenced by the fact that respondents were not native speakers. Differences between the models are associated with the specifics of their integration into the DSS.

The results prove the hypothesis that GenAI can improve the efficiency of DSSs by generating more useful but not more complex messages. These results also indicate that GenAI’s main advantage is in tailoring the DSS output to the needs of different user groups. The difference in results between the models highlights the need for proper testing of the developed AI solutions in specific contexts. The results will be used to develop a more efficient DSS for electricity prosumers.

Supporting Agency

  • The work is supported by the state budget scientific research project of Sumy State University, “Intelligent information technology for proactive management of energy infrastructure in conditions of risk and uncertainty” (state registration number 0123U101852).

Author Biographies

Oleh Lukianykhin, Sumy State University

PhD Student

Department of Information Technologies

Vira Shendryk, Sumy State University

PhD, Associate Professor, Head of Department

Department of Information Technologies

References

  1. Lopes, J. A. P., Madureira, A. G., Matos, M., Bessa, R. J., Monteiro, V., Afonso, J. L. et al. (2020). The future of power systems: Challenges, trends, and upcoming paradigms. WIREs Energy and Environment, 9 (3). https://doi.org/10.1002/wene.368
  2. Alotaibi, I., Abido, M. A., Khalid, M., Savkin, A. V. (2020). A Comprehensive Review of Recent Advances in Smart Grids: A Sustainable Future with Renewable Energy Resources. Energies, 13 (23), 6269. https://doi.org/10.3390/en13236269
  3. Peng, F. Z., Liu, C.-C., Li, Y., Jain, A. K., Vinnikov, D. (2024). Envisioning the Future Renewable and Resilient Energy Grids – A Power Grid Revolution Enabled by Renewables, Energy Storage, and Energy Electronics. IEEE Journal of Emerging and Selected Topics in Industrial Electronics, 5 (1), 8–26. https://doi.org/10.1109/jestie.2023.3343291
  4. Kröger, W., Nan, C.; Büscher, C., Schippl, J., Sumpf, P. (Eds.) (2018). Power systems in transition. Energy as a Sociotechnical Problem. London: Routledge, 41–78. https://doi.org/10.4324/9781315186313-3
  5. Marot, A., Kelly, A., Naglic, M., Barbesant, V., Cremer, J., Stefanov, A. et al. (2022). Perspectives on Future Power System Control Centers for Energy Transition. Journal of Modern Power Systems and Clean Energy, 10 (2), 328–344. https://doi.org/10.35833/mpce.2021.000673
  6. Machlev, R., Heistrene, L., Perl, M., Levy, K. Y., Belikov, J., Mannor, S. et al. (2022). Explainable Artificial Intelligence (XAI) techniques for energy and power systems: Review, challenges and opportunities. Energy and AI, 9, 100169. https://doi.org/10.1016/j.egyai.2022.100169
  7. Shin, D. (2020). User Perceptions of Algorithmic Decisions in the Personalized AI System:Perceptual Evaluation of Fairness, Accountability, Transparency, and Explainability. Journal of Broadcasting & Electronic Media, 64 (4), 541–565. https://doi.org/10.1080/08838151.2020.1843357
  8. Donida Labati, R., Genovese, A., Piuri, V., Scotti, F., Sforza, G. (2020). A Decision Support System for Wind Power Production. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50 (1), 290–304. https://doi.org/10.1109/tsmc.2017.2783681
  9. Sztubecka, M., Skiba, M., Mrówczyńska, M., Bazan-Krzywoszańska, A. (2020). An Innovative Decision Support System to Improve the Energy Efficiency of Buildings in Urban Areas. Remote Sensing, 12 (2), 259. https://doi.org/10.3390/rs12020259
  10. Badami, M., Fambri, G., Mancò, S., Martino, M., Damousis, I. G., Agtzidis, D. et al. (2019). A Decision Support System Tool to Manage the Flexibility in Renewable Energy-Based Power Systems. Energies, 13 (1), 153. https://doi.org/10.3390/en13010153
  11. Marot, A., Rozier, A., Dussartre, M., Crochepierre, L., Donnot, B. (2022). Towards an AI Assistant for Power Grid Operators. HHAI2022: Augmenting Human Intellect. IOS Press, 354, 79–95. https://doi.org/10.3233/faia220191
  12. Lyu, X., Li, Z., Ma, Q., She, M. (2023). Effects of accessible information amount and judgment times on human diagnostic performance of nuclear power plant faults. Ergonomics, 66 (7), 927–938. https://doi.org/10.1080/00140139.2022.2118836
  13. Fehrenbacher, D. D., Djamasbi, S. (2017). Information systems and task demand: An exploratory pupillometry study of computerized decision making. Decision Support Systems, 97, 1–11. https://doi.org/10.1016/j.dss.2017.02.007
  14. Allen, P. M., Edwards, J. A., Snyder, F. J., Makinson, K. A., Hamby, D. M. (2014). The Effect of Cognitive Load on Decision Making with Graphically Displayed Uncertainty Information. Risk Analysis, 34 (8), 1495–1505. https://doi.org/10.1111/risa.12161
  15. Phillips-Wren, G., Adya, M. (2020). Decision making under stress: the role of information overload, time pressure, complexity, and uncertainty. Journal of Decision Systems, 29 (1), 213–225. https://doi.org/10.1080/12460125.2020.1768680
  16. Afzal, U., Prouzeau, A., Lawrence, L., Dwyer, T., Bichinepally, S., Liebman, A. et al. (2022). Investigating Cognitive Load in Energy Network Control Rooms: Recommendations for Future Designs. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.812677
  17. Yacob, A., Roslim, A. D., Abdul Wahab, S. F., Abdul Halim, I. I., Baharum, Z., W. Hamzah, W. M. A. F. et al. (2024). Exploring the Landscape of Decision Support Systems: A Comprehensive Review of Implementations and Key Characteristics. Journal of Advanced Research in Applied Sciences and Engineering Technology, 51 (1), 141–149. https://doi.org/10.37934/araset.51.1.141149
  18. Moncur, B., Galvez Trigo, M. J., Mortara, L.; Schmorrow, D. D., Fidopiastis, C. M. (Eds.) (2023). Augmented Reality to Reduce Cognitive Load in Operational Decision-Making. Augmented cognition. HCII 2023. Lecture Notes in Computer Science. Cham: Springer, 328–346. https://doi.org/10.1007/978-3-031-35017-7_21
  19. Perry, N. C., Wiggins, M. W., Childs, M., Fogarty, G. (2013). The Application of Reduced-Processing Decision Support Systems to Facilitate the Acquisition of Decision-Making Skills. Human Factors: The Journal of the Human Factors and Ergonomics Society, 55 (3), 535–544. https://doi.org/10.1177/0018720812467367
  20. Phillips-Wren, G., Daly, M., Burstein, F. (2022). Support for cognition in decision support systems: an exploratory historical review. Journal of Decision Systems, 31 (1), 18–30. https://doi.org/10.1080/12460125.2022.2070946
  21. Stemmet, L., Ahmed, M. D.; Khosrow-Pour, M. (Ed.) (2014). The Decision Maker’s Cognitive Load. Encyclopedia of Information Science and Technology. IGI Global, 6466–6474. https://doi.org/10.4018/978-1-4666-5888-2.ch635
  22. Morrison, B. W., Kelson, J. N., Morrison, N. M. V., Innes, J. M., Zelic, G., Al-Saggaf, Y. et al. (2023). You’re Not the Boss of me, Algorithm: Increased User Control and Positive Implicit Attitudes Are Related to Greater Adherence to an Algorithmic Aid. Interacting with Computers, 35 (3), 452–460. https://doi.org/10.1093/iwc/iwad028
  23. Kanojiya, A., Nagori, V. (2018). Analysis of Architecture and Forms of Outputs of Decision Support Systems implemented for different domains. 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT). Coimbatore: IEEE, 346–350. https://doi.org/10.1109/icicct.2018.8472981
  24. Kelton, A. S., Pennington, R. R., Tuttle, B. M. (2010). The Effects of Information Presentation Format on Judgment and Decision Making: A Review of the Information Systems Research. Journal of Information Systems, 24 (2), 79–105. https://doi.org/10.2308/jis.2010.24.2.79
  25. Beriro, D., Nathanail, J., Salazar, J., Kingdon, A., Marchant, A., Richardson, S. et al. (2022). A Decision Support System to Assess the Feasibility of Onshore Renewable Energy Infrastructure. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3986807
  26. Taik, A., Nour, B., Cherkaoui, S. (2021). Empowering Prosumer Communities in Smart Grid with Wireless Communications and Federated Edge Learning. IEEE Wireless Communications, 28 (6), 26–33. https://doi.org/10.1109/mwc.017.2100187
  27. Stamatescu, I., Arghira, N., Făgărăşan, I., Stamatescu, G., Iliescu, S., Calofir, V. (2017). Decision Support System for a Low Voltage Renewable Energy System. Energies, 10 (1), 118. https://doi.org/10.3390/en10010118
  28. Ponraj, P., Murugesan, S. (2024). A Bi-level Decision Support System for Home Energy Management in Smart Homes. 2024 IEEE International Conference on Power Electronics, Drives and Energy Systems (PEDES). Mangalore: IEEE, 1–5. https://doi.org/10.1109/pedes61459.2024.10961886
  29. Gržanić, M., Capuder, T., Zhang, N., Huang, W. (2022). Prosumers as active market participants: A systematic review of evolution of opportunities, models and challenges. Renewable and Sustainable Energy Reviews, 154, 111859. https://doi.org/10.1016/j.rser.2021.111859
  30. Vachon, F., Lafond, D., Vallieres, B. R., Rousseau, R., Tremblay, S. (2011). Supporting situation awareness: A tradeoff between benefits and overhead. 2011 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA). Miami Beach: IEEE. https://doi.org/10.1109/cogsima.2011.5753460
  31. Lukianykhin, O., Shendryk, V., Shendryk, S., Malekian, R.; Karabegovic, I., Kovačević, A., Mandzuka, S. (Eds.) (2024). Promising AI Applications in Power Systems: Explainable AI (XAI), Transformers, LLMs. New Technologies, Development and Application VII. Cham: Springer, 66–76. https://doi.org/10.1007/978-3-031-66271-3_8
  32. Huynh, M.-T. (2024). Using generative AI as decision-support tools: unraveling users’ trust and AI appreciation. Journal of Decision Systems, 1–32. https://doi.org/10.1080/12460125.2024.2428166
  33. Yu, P., Xu, H., Hu, X., Deng, C. (2023). Leveraging Generative AI and Large Language Models: A Comprehensive Roadmap for Healthcare Integration. Healthcare, 11 (20), 2776. https://doi.org/10.3390/healthcare11202776
  34. Huang, Y., Kanij, T., Madugalla, A., Mahajan, S., Arora, C., Grundy, J. (2024). Unlocking Adaptive User Experience with Generative AI. Proceedings of the 19th International Conference on Evaluation of Novel Approaches to Software Engineering. Angers: SciTePress, 760–768. https://doi.org/10.5220/0012741000003687
  35. Iovane, G., Chinnici, M. (2024). Decision Support System Driven by Thermo-Complexity: Scenario Analysis and Data Visualization. Applied Sciences, 14 (6), 2387. https://doi.org/10.3390/app14062387
  36. Choi, S., Jain, R., Emami, P., Wadsack, K., Ding, F., Sun, H. et al. (2024). eGridGPT: Trustworthy AI in the Control Room. Office of Scientific and Technical Information (OSTI). https://doi.org/10.2172/2352232
  37. Lukianykhin, O., Shendryk, V. (2025). Machine learning-driven photovoltaic generation forecasting for prosumer decision support. Artificial Intelligence, 30 (1), 107–119. https://doi.org/10.15407/jai2025.01.107
  38. Huld, T., Ruf, H., Heilscher, G. (2014). Self-Consumption of Electricity by Households, Effects of PV System Size and Battery Storage. 29th European Photovoltaic Solar Energy Conference and Exhibition. Amsterdam, 4014–4017. https://doi.org/10.4229/EUPVSEC20142014-7AV.6.8
  39. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O. et al. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
  40. GPT-4 is OpenAI’s most advanced system, producing safer and more useful responses. OpenAI. Available at: https://openai.com/gpt-4
  41. Claude 3.5 Sonnet (2024). Anthropic. Available at: https://www.anthropic.com/news/claude-3-5-sonnet
  42. Fagbohun, O., Harrison, R. M., Dereventsov, A. (2023). An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner’s Guide. Journal of Artificial Intelligence, Machine Learning and Data Science, 1 (4), 1–11. https://doi.org/10.51219/jaimld/oluwole-fagbohun/15
  43. Beri, G., Srivastava, V. (2024). Advanced Techniques in Prompt Engineering for Large Language Models: A Comprehensive Study. 2024 IEEE 4th International Conference on ICT in Business Industry & Government (ICTBIG). Indore: IEEE, 1–4. https://doi.org/10.1109/ictbig64922.2024.10911672
  44. Moore, R. J., Arar, R. (2019). Conversational UX Design: A Practitioner’s Guide to the Natural Conversation Framework. ACM Books. https://doi.org/10.1145/3304087
  45. van Belle, G. (2008). Sample Size. Statistical Rules of Thumb. John Wiley & Sons, 27–51. https://doi.org/10.1002/9780470377963.ch2
  46. Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D. et al. (2020). SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature Methods, 17 (3), 261–272. https://doi.org/10.1038/s41592-019-0686-2
Use of generative artificial intelligence to improve output message effectiveness in decision support systems for prosumers

Downloads

Published

2025-08-29

How to Cite

Lukianykhin, O., & Shendryk, V. (2025). Use of generative artificial intelligence to improve output message effectiveness in decision support systems for prosumers. Technology Audit and Production Reserves, 4(2(84), 13–23. https://doi.org/10.15587/2706-5448.2025.333726

Issue

Vol. 4 No. 2(84) (2025): Information and control systems

Section

Information Technologies

License

Copyright (c) 2025 Oleh Lukianykhin, Vira Shendryk

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.