[1]王瑞群,欧阳权,段朝伟,等.基于强化学习的无人机全自主电力巡检[J].机械与电子,2021,(12):34-38.
　WANG Ruiqun,OUYANG Quan,DUAN Chaowei,et al.Autonomous Power Inspection of UAV Based on Reinforcement Learning[J].Machinery & Electronics,2021,(12):34-38.
点击复制

基于强化学习的无人机全自主电力巡检()
分享到：

机械与电子[ISSN:1001-2257/CN:52-1052/TH]

卷:
期数:
2021年12期
页码:
34-38
栏目:
自动控制与检测
出版日期:
2021-12-28

文章信息/Info

Title:
Autonomous Power Inspection of UAV Based on Reinforcement Learning
文章编号:
1001-2257 ( 2021 ) 12-0034-05
作者:
王瑞群; 欧阳权; 段朝伟; 王志胜
南京航空航天大学自动化学院,江苏 南京 211100
Author(s):
WANG Ruiqun ;  OUYANG Quan ;  DUAN Chaowei ;  WANG Zhisheng
( College of Automation Engineering , Nanjing University of Aeronautics and Astronautics , Nanjing 211100 , China )
关键词:
电力巡检; 强化学习; 近端策略优化; 无线充电; 能量优化
Keywords:
power line inspection ;  reinforcement learning ;  proximal policy optimization ;  wireless charging ;  energy optimization
分类号:
TP391 ; TM75
文献标志码:
A
摘要:
针对无人机在电力巡检中的全自主性进行研究,提出全自主电力巡检系统,该系统由无人机智能体、充电桩和待巡检目标构成。借助无线充电技术和强化学习决策算法使无人机拥有全自主执行任务的能力,并在设计的仿真环境中进行了验证,训练后的无人机智能体可以自主路径规划进行电力巡检和自主决策到附近的无线充电桩充电,无需人工介入可完成所有巡检任务。由于传统的近端策略优化算法在本系统中奖励低的问题,因此提出一种基于动作掩码的近端策略优化算法来训练无人机智能体,仿真实验结果表明,动作掩码机制使奖励提高了 39% ,能耗降低了 15.34% 。
Abstract:
Based on the research on the full autonomy of UAV in power line inspection , an autonomous power inspection system is proposed , which is composed of UAV agent , charging pile and targets to be inspected.With the aid of wireless charging technology and reinforcement learning algorithm , UAV has achieved the independent ability to perform a task , which has been verified in the design of simulation environment.Trained UAV agent can dopath planning for electric power inspection and make decision to nearby wireless charging pile autonomously , and all the inspection tasks can be done without human intervention.Due to low reward of traditional proximal strategy optimization ( PPO ) algorithm in this system , therefore , a kind of PPO algorithm based on action mask ( PPOAM ) is proposed to train UAV agent.The simulation results show that the action mask mechanism can increase the reward by 39% and reduce the energy consumption by 15.34%.

参考文献/References:

[ 1 ] 严波,林世忠,张振威,等 . 无人机电力巡检技术应用分析[ J ] . 自动化应用, 2019 ( 12 ): 155-156.

[ 2 ] 邢志伟,谭智炜,文涛,等 . 基于 DFS 的直升机电力巡检路径 规 划 [ J ] . 计 算 机 工 程 与 设 计,2021 , 42 ( 6 ):1779-1786.
[ 3 ] PATLE B K , PARHI D R K , JAGADEESH A , et al.Matrix-binary codes based genetic algorithm for path planning of mobile robot [ J ] .Computers and electrical engineering , 2018 , 67 : 708-728.
[ 4 ] MAVROVOUNIOTIS M , MULLER F M , YANG S X.Ant colony optimization with local search for dynamic traveling salesman problems [ J ] .IEEE Transactions on cybernetics , 2017 , 47 ( 7 ): 1743-1756.
[ 5 ] WEI X L , TANG C G , FAN J H , et al.Joint optimization of energy consumption and delay in cloud-to thing continuum [ J ] .IEEE Internet of things journal , 2019 , 6 ( 2 ): 2325-2337.?
[ 6 ] CAMPI T , CRUCIANI S , FELIZIANI M , et al.High efficiency and lightweight wireless charging system for drone batteries [ C ]// 2017 AEIT International Annual Conference , 2017 : 1-6.
[ 7 ] XIE J F , CARRILLO L R G , JIN L.Path planning for uav to cover multiple separated convex polygonal regions [ J ] .IEEE Access , 2020 , 8 : 51770-51785.
[ 8 ] LONG J , SHELHAMER E , DARRELL T.Fully convolutional networks for semantic segmentation [ C ]// 2015 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR ), 2015 : 3431-3440.
[ 9 ] SCHULMAN J , LEVINE S , ABBEEL P , et al.Trust region policy optimization [ C ]// International Conference on Machine Learning , 2015 : 1889-1897.
[ 10 ] KIM T Y , CHO S B.Predicting residential energy consumption using CNN-LSTM neural networks[ J ] .Energy , 2019 , 182 : 72-81.

相似文献/References:

[1]马占海,肖建斌,唐永成.基于激光 SLAM 技术的电力巡检机器人交互控制[J].机械与电子,2021,(12):72.
　MA Zhanhai,XIAO Jianbin,TANG Yongcheng.Interactive Control of Electric Power Inspection Robot Based on Laser SLAM Technology[J].Machinery & Electronics,2021,(12):72.
[2]王小龙,刘亨洋,张天毅,等.基于激光雷达的无人机路径规划研究[J].机械与电子,2023,41(08):56.
　WANG Xiaolong,LIU Hengyang,ZHANG Tianyi,et al.Research on Path Planning of UAV Based on Lidar[J].Machinery & Electronics,2023,41(12):56.
[3]郝 蛟,张宗包,邓 彬.基于地理信息云平台的变电站虚实交互技术研究[J].机械与电子,2023,41(09):13.
　HAO Jiao,ZHANG Zongbao,DENG Bin.Research on Virtual Reality Interaction Technology of Substation Based on Geographic Information Cloud Platform[J].Machinery & Electronics,2023,41(12):13.
[4]杨继阳,欧阳权,丛玉华,等.基于 Transformer 改进强化学习的无人机电力巡检规划[J].机械与电子,2024,42(10):54.
　YANG Jiyang,OUYANG Quan,CONG Yuhua,et al.UAV Power Inspection Planning Based on Transformer Improved Reinforcement Learning[J].Machinery & Electronics,2024,42(12):54.

备注/Memo

备注/Memo:
收稿日期: 2021-08-16
基金项目:江苏省高校自然科学研究面上项目( 18KJB520023 );南京航空航天大学研究生创新竞赛( 016001 )
作者简介:王瑞群 ( 1999- ),男,河北邯郸人,硕士研究生,研究方向为深度强化学习与无人机飞行控制;欧阳权 ( 1991- ),男,湖北仙桃人,博士,讲师,研究方向为无人机飞行控制、电池管理等;段朝伟 ( 1983- ),男,河南禹州人,博士,讲师,研究方向为图像处理、机器视觉和模式识别等;王志胜 ( 1970- ),男,湖北荆门人,博士,教授,研究方向为信息融合、无人机蜂群控制和计算机视觉等。
更新日期/Last Update: 2021-12-28