APPROACH TO BUILDING A GLOBAL MOBILE AGENT WAY BASED ON Q-LEARNING
DOI:
https://doi.org/10.30837/ITSSI.2020.13.043Keywords:
path planning, Q-learning, mobile works, adaptive standalone search algorithmsAbstract
Today, the problem of navigation of autonomous mobile systems in a space where disturbances are possible is urgent. The task of finding a route for a mobile robot is a complex and non-trivial task. At the moment, there are many algorithms that allow you to solve such problems in accordance with the specified criteria for building a route. Most of these algorithms are modifications of "basic" path planning methods that are optimized for specific conditions. The subject of research in the article is the process of building a global path for a mobile agent. The purpose of the work is to create an algorithm for planning the route of autonomous mobile systems in space using the Q-learning algorithm. The following tasks are solved in the article: development of an approach to training and support of a reinforcement learning algorithm for building a global path of a mobile agent; testing the agent's ability to find a path in environments that are not in the training set. The following methods are used: graph theory, queuing theory, Markov decision-making process theory and mathematical programming methods. The research is based on scientific articles and other materials from foreign conferences and archives in the field of machine learning, deep learning and deep reinforcement learning. The following results were obtained: an approach was formulated to construct the global path of a mobile agent based on the accumulated data in the process of interaction with the external environment. The environment rewards these actions and the agent continues to carry them out. This approach will allow this method to be applied to a wide range of situations and devices. Conclusions: This approach allows accumulating the knowledge of the outside world for further decision-making when planning a route where the robot can acquire the skill of self-learning, studying and training like a human, and finding the path from the initial state to the target state in an unknown environment. In the modern world, the use of robots and autonomous systems is spreading, designed to replace or facilitate human labor, make it safer and speed it up. Adaptive autonomous path finding algorithms are very important in many robotics applications. Thus, navigation tasks with limited information are relevant today, since this is the main task that the agent solves, and one of the tasks that are part of the robot during operation.
References
Kargin, A., Ivaniuk, O. (2020), "Autonomous robot motion control situational planning model", Advanced Information Systems, No 4, P. 41–51. DOI: 10.20998/2522-9052.2020.3.05
Sukharev, O. (2019), "Functions of information and modes of informational development of control systems", Problems of theory and practice of management, No 1, P. 37–51.
Miyazawa, K. (2002), "Fire robots developed by the Tokyo Fire Department", Advanced Robotics, P. 553–556. DOI: 10.1163/156855302320535953
Magid, E., Lavrenov, R., Afanasyev, I. (2017), "Voronoi-based trajectory optimization for UGV path planning", International Conference on Mechanical, System and Control Engineering (ICMSC), IEEE, P. 383–387. DOI: 10.1109/ICMSC.2017.7959506
Kovács, G., Yusupova, N., Smetanina, O., Rassadnikova, E. (2018), "Methods and algorithms to solve the vehicle routing problem with time windows and further conditions", Pollack Periodica, No. 13 (1), P. 65–76. DOI: 10.1556/606.2018.13.1.6
Lavrenov, R. O., Magid, E. A. (2020), "Multihomotopic search for the optimal route for autonomous mobile devices", Industrial automation, No. 7, P. 61–64. DOI: 10.25728/avtprom.2020.07.14
Hassan, A. M., Elias, C. M., Shehata, O. M., Morgan, E. I. (2017), "A global integrated artificial potential field/virtual obstacles path planning algorithm for multi-robot system applications", Int. Research J. of Eng. and Technology, No. 4 (9), P. 1198–1204.
Wahid, N., Zamzuri, H., Amer, N. H., Dwijotomo, A., Saruchi, S. A., Mazlan, S. A. (2020), "Vehicle collision avoidance motion planning strategy using artificial potential field with adaptive multi-speed scheduler", IET Intelligent Transport Systems, No. 14 (10), P. 1200–1209. DOI: 10.1049/iet-its.2020.0048
Ku Ping Cheng, Rajesh Elara Mohan, Nguyen Huu Khanh Nhan, Anh Vu Le (2020), "Multi-Objective Genetic Algorithm-Based Autonomous PP for Hinged-Tetro Reconfigurable Tiling Robot", IEEE Access, available at : https://ieeexplore.ieee.org/abstract/document/9131750 (last accessed: 23.09.2020)
Che Gaofeng, Lijun Liu, Zhen Yu (2020), "An improved ant colony optimization algorithm based on particle swarm optimization algorithm for path planning of autonomous underwater vehicle", Journal of Ambient Intelligence and Humanized Computing, No. 11 (8), P. 3349–3354. DOI: 10.1007/s12652-019-01531-8
Raja, P., Pugazhenthi, S. (2012), "Optimal path planning of mobile robots: A review", International journal of physical sciences, No. 7 (9), P. 1314–1320. DOI: 10.5897/IJPS11.1745
Yahja, A. (1998), "Framed-quadtree path planning for mobile robots operating in sparse environments", In: Proceedings. IEEE International Conference on Robotics and Automation (Cat. No. 98CH36146), P. 650–655. DOI: 10.1109/ROBOT.1998.677046
Montiel, O., Orozco-Rosas, U., Sepúlveda, R. (2015), "Path planning for mobile robots using Bacterial Potential Field for avoiding static and dynamic obstacles", Expert Systems with Applications, No. 42 (12), P. 5177–5191. DOI: 10.1016/j.eswa.2015.02.033
Raja, P., Pugazhenthi, S. (2009), "Path planning for mobile robots in dynamic environments using particle swarm optimization", 2009 International Conference on Advances in Recent Technologies in Communication and Computing. IEEE, P. 401–405. DOI: 10.1109/ARTCom.2009.24
Kovács, B., Szayer, G., Tajti, F., Burdelis, M., Korondi, P. (2016), "A novel potential field method for path planning of mobile robots by adapting animal motion attributes", Robotics and Autonomous Systems, No. 82, P. 24–34. DOI: 10.1016/j.robot.2016.04.007
Downloads
How to Cite
Issue
Section
License
Copyright (c) 2020 Vitalii Martovytskyi, Oleksandr Ivaniuk
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Our journal abides by the Creative Commons copyright rights and permissions for open access journals.
Authors who publish with this journal agree to the following terms:
Authors hold the copyright without restrictions and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-commercial and non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
Authors are permitted and encouraged to post their published work online (e.g., in institutional repositories or on their website) as it can lead to productive exchanges, as well as earlier and greater citation of published work.