Trajectory planning and resource allocation optimization in UAV data collection missions
-
摘要:
针对无人机（UAV）在应急场景中执行数据采集任务时，其电池容量有限、缓存空间有限，以及地面目标优先级动态变化所导致的无人机航迹规划与资源分配效果较差的问题，提出一种基于深度强化学习的无人机航迹规划与资源分配联合优化方法。考虑无人机任务中的通信、计算、飞行、数据缓存过程，构建相应的数学问题模型；针对无人机航迹规划与资源分配问题构建马尔可夫过程模型，设计相应的状态和行为描述及用于平衡无人机能耗和数据采集信息量的加权奖励函数；与贪婪算法和遗传算法等智能优化方法进行仿真对比。结果表明：所提方法能够使无人机在较短的任务时间内，以消耗相似或者较低的能量为代价，较大提升对地面用户的数据采集量。
Abstract:A joint optimization method for unmanned aerial vehicle (UAV) trajectory planning and resource allocation based on deep reinforcement learning was proposed to address the challenges of limited battery capacity, limited cache space, and dynamic changes in ground target priorities during data collection tasks in emergency scenarios. First, a mathematical model was developed by considering the communication, computation, flight, and data caching processes in UAV missions. Then, a Markov process model was established for UAV trajectory planning and resource allocation, with corresponding state and action descriptions. A weighted reward function was designed to balance UAV energy consumption and data collection volume. Finally, simulations were conducted to compare the proposed method with greedy algorithms and genetic algorithms. The results show that the proposed method can significantly improve the amount of data collected from ground users within a shorter task time, at a similar or lower energy cost for UAVs.
-
图 2 无人机数据采集任务环境模型结构
Figure 2. Environmental model structure for UAV data collection tasks
图 4 不同算法总奖励均值对比
Figure 4. Comparison of average total rewards for different algorithms
图 5 不同算法的总能耗均值对比
Figure 5. Comparison of average total energy consumption for different algorithms
图 6 不同算法的采集总信息量均值对比
Figure 6. Comparison of average total information collected for different algorithms
图 7 不同能耗惩罚系数下不同算法总奖励均值对比
Figure 7. Comparison of total rewards for different algorithms under varying energy consumption penalty coefficients
图 8 不同能耗惩罚系数下不同算法总能耗均值对比
Figure 8. Comparison of total energy consumption for different algorithms under varying energy consumption penalty coefficients
图 9 不同能耗惩罚系数下不同算法采集总信息量均值对比
Figure 9. Comparison of total information collected for different algorithms under varying energy consumption penalty coefficients
表 1 地面目标优先级动态转移矩阵
Table 1. Ground target priority dynamic transfer matrix
优先级 优先级概率 Ⅰ Ⅱ Ⅲ Ⅳ Ⅰ 0.8 0.1 0.08 0.02 Ⅱ 0 0.8 0.16 0.04 Ⅲ 0 0 0.9 0.1 Ⅳ 0 0 0 1 
下载: 导出CSV
表 2 各优先级目标的信息量
Table 2. Information volume of targets with different priorities
优先级 信息量 Ⅰ 1 Ⅱ 2 Ⅲ 3 Ⅳ 4
下载: 导出CSV
表 3 无人机航迹规划与资源分配联合优化方法参数
Table 3. Parameters for joint optimization algorithm in UAV trajectory planning and resource allocation
算法 种群测试回合数 种群大小 迭代次数 学习率 网络结构 缓存池大小 批大小 折扣因子 软更新系数 训练步数 遗传算法 100 200 1000 SAC算法 3×10−4 128×128×128 500000 256 0.99 0.005 5000000 贪婪算法 100 200
下载: 导出CSV
表 4 不同算法在不同初始缓存下的性能对比
Table 4. Performance comparison of different algorithms with varying initial cache sizes
算法 缓存/GB 奖励 能耗/kJ 采集信息量 遗传算法 8 52.79±3.26 83.10±0 61.10±3.27 16 55.04±3.02 64.40±0 61.48±3.02 32 53.18±3.31 48.95±0 58.08±3.31 64 55.37±3.42 37.82±0 59.15±3.42 128 54.77±3.42 36.32±0 58.40±3.42 贪婪算法 8 36.39±3.29 143.32±23.10 50.72±3.87 16 46.56±3.71 172.39±18.79 63.80±3.73 32 47.10±3.93 179.48±20.72 65.05±3.52 64 49.11±3.41 119.23±15.29 61.03±3.04 128 50.90±3.03 96.51±14.85 61.55±3.11 SAC算法 8 62.10±2.42 88.86±6.58 70.97±2.33 16 64.83±2.93 71.29±13.10 71.96±2.28 32 66.25±1.91 59.17±5.51 72.17±1.86 64 63.12±2.79 45.44±8.72 67.66±2.19 128 63.24±2.14 40.08±5.89 67.24±2.16
下载: 导出CSV
-
[1] 胡欣颖. 面向信息采集场景的无人机轨迹规划与通信资源分配研究[D]. 北京: 北京邮电大学, 2021: 1-3.HU X Y. Research on UAV trajectory planning and communication resource allocation for information acquisition scenes[D]. Beijing: Beijing University of Posts and Telecommunications, 2021: 1-3(in Chinese). [2] LI Z R, ZHANG Y W, WU H, et al. Design and application of a UAV autonomous inspection system for high-voltage power transmission lines[J]. Remote Sensing, 2023, 15(3): 865. doi: 10.3390/rs15030865 [3] DONG M X, OTA K, LIN M, et al. UAV-assisted data gathering in wireless sensor networks[J]. The Journal of Supercomputing, 2014, 70(3): 1142-1155. doi: 10.1007/s11227-014-1161-6 [4] GONG J, CHANG T H, SHEN C, et al. Flight time minimization of UAV for data collection over wireless sensor networks[J]. IEEE Journal on Selected Areas in Communications, 2018, 36(9): 1942-1954. doi: 10.1109/JSAC.2018.2864420 [5] CHEN X, CHEN X M. The UAV dynamic path planning algorithm research based on Voronoi diagram[C]//Proceedings of the 26th Chinese Control and Decision Conference. Piscataway: IEEE Press, 2014: 1069-1071. [6] HSU D, LATOMBE J C, KURNIAWATI H. On the probabilistic foundations of probabilistic roadmap planning[J]. International Journal of Robotics Research, 2006, 25(7): 627-643. [7] LIU X. Four alternative patterns of the Hilbert curve[J]. Applied Mathematics and Computation, 2004, 147(3): 741-752. doi: 10.1016/S0096-3003(02)00808-1 [8] MU X D, LIU Y W, GUO L, et al. Energy-constrained UAV data collection systems: NOMA and OMA[J]. IEEE Transactions on Vehicular Technology, 2021, 70(7): 6898-6912. doi: 10.1109/TVT.2021.3086556 [9] SAMIR M, SHARAFEDDINE S, ASSI C M, et al. UAV trajectory planning for data collection from time-constrained IoT devices[J]. IEEE Transactions on Wireless Communications, 2020, 19(1): 34-46. doi: 10.1109/TWC.2019.2940447 [10] ZHAN C, LAI H. Energy minimization in internet-of-things system based on rotary-wing UAV[J]. IEEE Wireless Communications Letters, 2019, 8(5): 1341-1344. doi: 10.1109/LWC.2019.2916549 [11] NGUYEN K K, DUONG T Q, DO-DUY T, et al. 3D UAV trajectory and data collection optimisation via deep reinforcement learning[J]. IEEE Transactions on Communications, 2022, 70(4): 2358-2371. doi: 10.1109/TCOMM.2022.3148364 [12] FU S, TANG Y J, WU Y, et al. Energy-efficient UAV-enabled data collection via wireless charging: a reinforcement learning approach[J]. IEEE Internet of Things Journal, 2021, 8(12): 10209-10219. doi: 10.1109/JIOT.2021.3051370 [13] BAYERLEIN H, THEILE M, CACCAMO M, et al. UAV path planning for wireless data harvesting: a deep reinforcement learning approach[C]//Proceedings of the IEEE Global Communications Conference. Piscataway: IEEE Press, 2020: 1-6. [14] HAARNOJA T, ZHOU A, ABBEEL P, et al. Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor[C]//Proceedings of the International Conference on Machine Learning. Piscataway: IEEE Press, 2018: 1861-1870. [15] ISLAM S K, HAIDER M R. Sensors and low power signal processing[M]. Berlin: Springer Science & Business Media, 2009. [16] LIU Q, SHI L, SUN L L, et al. Path planning for UAV-mounted mobile edge computing with deep reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2020, 69(5): 5723-5728. doi: 10.1109/TVT.2020.2982508 [17] WU Q Q, ZENG Y, ZHANG R. Joint trajectory and communication design for multi-UAV enabled wireless networks[J]. IEEE Transactions on Wireless Communications, 2018, 17(3): 2109-2121. doi: 10.1109/TWC.2017.2789293 [18] LAMBORA A, GUPTA K, CHOPRA K. Genetic algorithm-a literature review[C]//Proceedings of the International Conference on Machine Learning, Big Data, Cloud and Parallel Computing. Piscataway: IEEE Press, 2019: 380-384. -
下载:
点击查看大图
计量
- 文章访问数: 250
- HTML全文浏览量: 68
- PDF下载量: 15
- 被引次数: 0
