Design and development of a large language model-based tool for vulnerability detection
DOI:
https://doi.org/10.15587/1729-4061.2025.325251Keywords:
large language models, vulnerability detection automation, artificial intelligence, multi-vector testingAbstract
The subject of this study is a tool for automating vulnerability detection using large language models, developed to reduce the time spent on conventional penetration testing. In addition, a detailed analysis has been conducted comparing the effectiveness of the automated approach with that of conventional manual security testing. The tool utilizes application programming interface access to LLMs, enabling the analysis of large volumes of data, the identification of complex relationships between system components, and the provision of interactive support to specialists during the testing process. By conducting experiments under actual conditions, the tool demonstrated the ability to integrate with popular penetration test tools and deal with real cyber threats, particularly in scenarios involving active attacks on networks and web applications. By automating routine tasks, such as configuration checks, analysis of tool outputs, and generating recommendations, the tool significantly reduces the workload on specialists. On average, the tool shortened the testing time by 54.4 % compared to a manual approach. Recall reached 94.7 % in network analysis scenarios but dropped to 66.7 % in web application testing, while the automated approach’s precision ranged from 80 % to 90 %. The study results confirmed that the application of large language models in the penetration testing process significantly reduces the time required to complete tasks and improves the accuracy of vulnerability detection. The tool could be used both independently and in combination with other automation tools, making it a versatile solution for organizations of various sizes. Thus, the proposed solution is a substantial contribution to the development of modern cybersecurity technologies and demonstrates the prospects of integrating artificial intelligence into automation processes
References
- Tolkachova, A., Piskozub, A. (2024). Methods for testing the security of web applications. Electronic Professional Scientific Journal «Cybersecurity: Education, Science, Technique», 2 (26), 115–122. https://doi.org/10.28925/2663-4023.2024.26.668
- Li, Z., Dutta, S., Naik, M. (2024). LLM-assisted static analysis for detecting security vulnerabilities. arXiv. https://doi.org/10.48550/arXiv.2405.17238
- Saini, J., Bansal, A. (2024). Automated penetration testing: machine learning approach. CEUR Workshop Proceedings. Available at: https://ceur-ws.org/Vol-3682/Paper10.pdf
- Omar, M. (2023). Detecting software vulnerabilities using language models. arXiv. https://doi.org/10.48550/arXiv.2302.11773
- Purba, M. D., Ghosh, A., Radford, B. J., Chu, B. (2023). Software Vulnerability Detection using Large Language Models. 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW), 112–119. https://doi.org/10.1109/issrew60843.2023.00058
- Sultana, S., Afreen, S., Eisty, N. U. (2024). Code vulnerability detection: A comparative analysis of emerging large language models. arXiv. https://doi.org/10.48550/arXiv.2409.10490
- Goyal, D., Subramanian, S., Peela, A. (2024). Hacking, the lazy way: LLM augmented pentesting. arXiv. https://doi.org/10.48550/arXiv.2409.09493
- Pratama, D., Suryanto, N., Adiputra, A. A., Le, T.-T.-H., Kadiptya, A. Y., Iqbal, M., Kim, H. (2024). CIPHER: Cybersecurity Intelligent Penetration-Testing Helper for Ethical Researcher. Sensors, 24 (21), 6878. https://doi.org/10.3390/s24216878
- Aloraini, B., Nagappan, M., German, D. M., Hayashi, S., Higo, Y. (2019). An empirical study of security warnings from static application security testing tools. Journal of Systems and Software, 158, 110427. https://doi.org/10.1016/j.jss.2019.110427
- Singh, R., Kumar Gupta, M., Patil, D. R., Maruti Patil, S. (2024). Analysis of Web Application Vulnerabilities using Dynamic Application Security Testing. 2024 IEEE 9th International Conference for Convergence in Technology (I2CT), 1–6. https://doi.org/10.1109/i2ct61223.2024.10543484
- Mallissery, S., Wu, Y.-S. (2023). Demystify the Fuzzing Methods: A Comprehensive Survey. ACM Computing Surveys, 56 (3), 1–38. https://doi.org/10.1145/3623375
- Khaliq, S., Abideen Tariq, Z. U., Masood, A. (2020). Role of User and Entity Behavior Analytics in Detecting Insider Attacks. 2020 International Conference on Cyber Warfare and Security (ICCWS), 1–6. https://doi.org/10.1109/iccws48432.2020.9292394
- Mohammed, F., Rahman, N. A. A., Yusof, Y., Juremi, J. (2022). Automated Nmap Toolkit. 2022 International Conference on Advancements in Smart, Secure and Intelligent Computing (ASSIC), 1–7. https://doi.org/10.1109/assic55218.2022.10088375
- Choudhary, R., Rawat, J., Singh, G. (2023). Comprehensive Exploration of Web Application Security Testing with Burp Suite Tools. International Journal For Multidisciplinary Research, 5 (6). https://doi.org/10.36948/ijfmr.2023.v05i06.11297
- Narayana Rao, T. V., Shravan, V. (2019). Metasploit Unleashed Tool for Penetration Testing. International Journal on Recent and Innovation Trends in Computing and Communication, 7 (4), 16–20. https://doi.org/10.17762/ijritcc.v7i4.5285
- Suga, Y. (2014). Visualization of SSL Setting Status Such as the FQDN Mismatch. 2014 Eighth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, 588–593. https://doi.org/10.1109/imis.2014.88
- Bhoir, H., Jayamalini, K. (2021). Web Crawling on News Web Page using Different Frameworks. International Journal of Scientific Research in Science and Technology, 513–519. Internet Archive. https://doi.org/10.32628/cseit2174120
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Anastasiia Zhuravchak, Andrian Piskozub, Bohdan Skorynovych, Yuriy Lakh, Danyil Zhuravchak, Pavlo Hlushchenko, Petro Venherskyi, Igor Beliaiev, Maksym Vorokhob, Ivan Kolbasynskyi
This work is licensed under a Creative Commons Attribution 4.0 International License.
The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.
A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.
