Exploring the power of heterogeneous UAV swarms through reinforcement learning
DOI:
https://doi.org/10.15587/2706-5448.2023.293063Keywords:
reinforcement learning, robot swarms, heterogeneous swarms, UAV swarms, heterogeneous UAV swarmsAbstract
The object of research is heterogeneous and homogeneous swarms of unmanned aerial vehicles (UAVs). The primary focus of this study is the comparison between heterogeneous and homogeneous UAV swarms, examining their performance in a simulated environment designed using the Python Gym library. The research involves implementing reinforcement learning algorithms, specifically the Proximal Policy Optimization (PPO), to train and evaluate the swarms.
The central issue addressed by this research is to determine which type of UAV swarm – heterogeneous or homogeneous – exhibits better performance in a defined task. The chosen task involves searching for groups of objects in an unknown area, emphasizing the ability of the swarm to adapt and efficiently locate objects in dynamic environments.
The obtained results reveal an advantage for heterogeneous UAV swarms over their homogeneous counterparts. The heterogeneous swarm has a steeper learning curve and achieves higher rewards in fewer episodes during the training phase. The key finding indicates that the varied skill set within the heterogeneous swarm allows for quicker adaptation to changing environmental conditions. The superior performance of the heterogeneous swarm is attributed to the diversity of capabilities among its UAV agents, enabling them to leverage their individual strengths to achieve better overall performance in the given task.
The practical application of these results is contingent upon the task requirements and environmental conditions. In scenarios where tasks demand diverse skills and adaptability to changing conditions, heterogeneous UAV swarms are recommended. The results suggest their efficacy in applications such as search and rescue operations, environmental monitoring, and other dynamic tasks.
In conclusion, this research provides valuable insights into optimizing UAV swarm composition for specific tasks. The results contribute both theoretically and practically by highlighting the advantages of heterogeneity in swarm capabilities.
References
- Telli, K., Kraa, O., Himeur, Y., Ouamane, A., Boumehraz, M., Atalla, S., Mansoor, W. (2023). A Comprehensive Review of Recent Research Trends on Unmanned Aerial Vehicles (UAVs). Systems, 11 (8), 400. doi: https://doi.org/10.3390/systems11080400
- Abdelkader, M., Güler, S., Jaleel, H., Shamma, J. S. (2021). Aerial Swarms: Recent Applications and Challenges. Current Robotics Reports, 2 (3), 309–320. doi: https://doi.org/10.1007/s43154-021-00063-4
- Bushnaq, O. M., Mishra, D., Natalizio, E., Akyildiz, I. F. (2022). Unmanned aerial vehicles (UAVs) for disaster management. Nanotechnology-Based Smart Remote Sensing Networks for Disaster Prevention. Elsevier Inc., 159–188. doi: https://doi.org/10.1016/b978-0-323-91166-5.00013-6
- Qu, C., Boubin, J., Gafurov, D., Zhou, J., Aloysius, N., Nguyen, H., Calyam, P. (2022). UAV Swarms in Smart Agriculture: Experiences and Opportunities. 2022 IEEE 18th International Conference on E-Science (e-Science). doi: https://doi.org/10.1109/escience55777.2022.00029
- Ming, R., Jiang, R., Luo, H., Lai, T., Guo, E., Zhou, Z. (2023). Comparative Analysis of Different UAV Swarm Control Methods on Unmanned Farms. Agronomy, 13 (10), 2499. doi: https://doi.org/10.3390/agronomy13102499
- Wu, M., Zhu, X., Ma, L., Wang, J., Bao, W., Li, W., Fan, Z. (2022). Torch: Strategy evolution in swarm robots using heterogeneous-homogeneous coevolution method. Journal of Industrial Information Integration, 25, 100239. doi: https://doi.org/10.1016/j.jii.2021.100239
- Ferrante, E. (2009). A Control Architecture for a Heterogeneous Swarm of Robots The Design of a Modular Behavior-based Architecture. IRIDIA – Technical Report Series. Available at: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=06c577a024d37f6a1cab79c74368ec54b8837880
- Dorigo, M., Floreano, D., Gambardella, L. M., Mondada, F., Nolfi, S., Baaboura, T. et al. (2013). Swarmanoid: A Novel Concept for the Study of Heterogeneous Robotic Swarms. IEEE Robotics & Automation Magazine. Available at: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=09db526ea45bf5829f049f69100eb86322fe44fb
- Price, I. C. (2006). Evolving Self-Organized Behavior for Homogeneous and Heterogeneous UAV or UCAV Swarms. USAF. Air Force Institute of Technology. Available at: https://scholar.afit.edu/cgi/viewcontent.cgi?article=4466&context=etd
- Albrekht, Y., Pysarenko, A. (2023). Unknown location targets searching system in known environment using reinforcement learning. Adaptive systems of automatic control, 1 (42), 9–14. doi: https://doi.org/10.20535/1560-8956.42.2023.278920
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Yosyp Albrekht, Andrii Pysarenko
This work is licensed under a Creative Commons Attribution 4.0 International License.
The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.
