基于混合注意力的全天候机场跑道异物检测

张敬博; 任杰; 王美琪

doi:10.13700/j.bh.1001-5965.2023.0500

基于混合注意力的全天候机场跑道异物检测

doi: 10.13700/j.bh.1001-5965.2023.0500

石家庄铁道大学机械工程学院，石家庄 050043

基金项目:

国家自然科学基金(12102273)

详细信息

通讯作者:
E-mail：renjiebeijing@163.com

中图分类号: TP391.41；V328
计量
- 文章访问数: 261
- HTML全文浏览量: 77
- PDF下载量: 25
- 被引次数: 0
出版历程
- 收稿日期: 2023-07-29
- 录用日期: 2023-09-22
- 网络出版日期: 2023-10-18
- 整期出版日期: 2025-09-30

All-weather airport runway foreign object debris detection based on mixed attention

School of Mechanical Engineering，Shijiazhuang Tiedao University，Shijiazhuang 050043，China

Funds:

National Natural Science Foundation of China (12102273)

More Information

Corresponding author: E-mail：renjiebeijing@163.com

摘要

摘要:
机场跑道的异物检测对于飞机安全起降起着至关重要的作用，然而不同光照、天气跑道环境下现有的检测算法存在漏检、误检等现象，为此提出了一种适用于全天候机场跑道环境的YOLOv5异物检测算法。针对原有网络中池化过程存在一定程度的特征损失问题，设计了跨阶段局部空间金字塔池化模块，可以自适应提取深层特征语义信息，增强网络多尺度表征能力；在特征融合部分引入混合注意力模块，利用通道和空间特征权值再分配，强化异物和无关背景要素的特征差异；针对小目标异物难以识别和定位从而导致漏检现象，设计了多尺度定位损失函数，通过加入相似性度量提高小目标的检测能力；使用优化后的训练策略训练MS-FOD数据集。实验结果表明：改进的算法达到了95.83%的均值平均精度和94.31%的召回率，相比原始的YOLOv5分别提高了3.68%和15.69%，同时检测速度为68帧/s，满足实时性异物检测的需求，有效验证了所提算法对机场跑道异物检测的有效性。
- 机场跑道异物 /
- YOLOv5 /
- 跨阶段局部结构 /
- 混合注意力机制 /
- 多尺度损失函数
Abstract:
The foreign object debris (FOD) detection of airport runway plays an important role in the safe take-off and landing of aircraft. However, the existing detection algorithms in different light and weather runway environments have the phenomenon of missed detection and false detection. Therefore, a YOLOv5 FOD detection algorithm suitable for all-weather airport runway environments is proposed. Firstly, aiming at the problem of feature loss in the pooling process of the original network, a cross stage partial spatial pyramid pooling module is designed, which can adaptively extract deep feature semantic information and enhance the multiscale representation ability of the network. Secondly, the mixed attention module is introduced in the feature fusion part, and the channel and spatial feature weights are redistributed to strengthen the feature differences between FOD and unrelated background elements. Then, a multiscale positioning loss function is designed to improve the detection ability of small targets by adding similarity measures, aiming at the phenomenon that small target FOD are difficult to identify and locate, which leads to missed detection. Finally, the optimized training strategy is used to train the MS-FOD dataset. The experimental results show that the improved algorithm achieves an average accuracy of 95.83%and a recall rate of 94.31%, which is 3.68 and 15.69 percentage points higher than the original YOLOv5, respectively. At the same time, the detection speed FPS is 68 frames per second, which meets the needs of real-time FOD detection. The effectiveness of the proposed algorithm for airport runway FOD detection is effectively verified.
- foreign object debris on airport runways /
- YOLOv5 /
- cross stage partial structure /
- mixed attention mechanism /
- multiscale loss function

HTML全文

图 1 本文算法的整体框架

Figure 1. The overall framework of our algorithm

下载: 全尺寸图片幻灯片

图 2 P-SPPCSPC模块

Figure 2. P-SPPCSPC module

下载: 全尺寸图片幻灯片

图 3 部分卷积和标准卷积的卷积过程

Figure 3. The convolutional process of partial convolution and convolution

下载: 全尺寸图片幻灯片

图 4 CBAM注意力模块

Figure 4. CBAM attention module

下载: 全尺寸图片幻灯片

图 5 各类别的实例数量

Figure 5. Number of instances for each category

下载: 全尺寸图片幻灯片

图 6 MS-FOD数据集部分标注示意

Figure 6. Partial annotation diagram of MS-FOD dataset

下载: 全尺寸图片幻灯片

图 7 优化后的Mosaic数据增强示意

Figure 7. Optimized Mosaic data enhancement diagram

下载: 全尺寸图片幻灯片

图 8 训练过程中各算法mAP对比

Figure 8. The mAP comparison of each algorithm in the training process

下载: 全尺寸图片幻灯片

图 9 改进前后多种场景下的效果对比

Figure 9. Comparison of effects in various environments before and after improvement

下载: 全尺寸图片幻灯片

表 1 初始训练参数

Table 1. Initial training parameters

参数	数值
输入图片尺寸/像素	640×640
批处理大小	16
优化器	SGD
初始学习率	0.01
动量	0.937
权重衰减	0.000 5
训练轮数	300

下载: 导出CSV

表 2 聚类前后先验框尺寸

Table 2. Prior box size before and after clustering

检测层	默认先验框	聚类后的先验框
P₃	(10,13), (16,30), (33,23)	(11,11), (17,17), (26,14)
P₄	(30,61), (62,45), (59,119)	(23,24), (37,35), (78,28)
P₅	(116,90), (156,198), (373,326)	(31,74), (66,56), (92,95)

下载: 导出CSV

表 3 不同池化模块效果对比

Table 3. Effect comparison of different pooling modules

模型	空间金字塔池化模块	mAP/%	参数量
基线模型	SPP	92.15	7.2×10⁶
模型1	SPPCSPC(Conv)	93.43(+1.28)	13.5×10⁶
模型2	SPPCSPC(GConv,4groups)	92.88(+0.73)	8.2×10⁶
模型3	SPPCSPC(DWConv)	93.62(+1.47)	9.3×10⁶
模型4	P-SPPCSPC	93.87(+1.72)	9.1×10⁶

下载: 导出CSV

表 4 不同损失函数效果对比

Table 4. Effect comparison of different loss functions

损失函数	P/%	R/%	mAP/%
GIoU	95.66	78.62	92.15
DIoU	95.27	78.83	92.24(+0.09)
CIoU	95.04	79.29	92.39(+0.24)
NC-Loss(0.3)	94.53	81.69	92.21(+0.07)
NC-Loss(0.5)	93.47	83.99	92.58(+0.43)
NC-Loss(0.7)	93.30	88.83	93.57(+1.42)
NWD	89.41	90.27	91.69(-0.46)

下载: 导出CSV

表 5 消融实验结果

Table 5. Ablation experimental results

模型	训练策略优化	P_SPPCSPC	CBAM	NC-Loss	mAP/%	mAP_S/%	mAP_M/%	mAP_L/%	检测速度/(帧·s⁻¹)	参数量
基线模型					92.15	85.78	90.59	95.52	80	7.2×10⁶
G1	√				92.81(+0.66)	87.39	92.35	95.89	80	7.2×10⁶
G2	√	√			94.34(+1.53)	91.42	94.77	96.18	75	9.2×10⁶
G3	√	√	√		94.93(+0.59)	92.56	95.51	96.58	68	9.3×10⁶
G4	√	√	√	√	95.83(+0.90)	94.44	95.98	96.46	68	9.3×10⁶

下载: 导出CSV

表 6 不同算法模型的检测结果对比

Table 6. Comparison of detection results of different algorithm models

模型	主干网络	P/%	R/%	mAP/%	检测速速/(帧·s⁻¹)	参数量/M
Faster R-CNN	ResNet50	69.67	91.90	83.54	11	136.9×10⁶
SSD	VGG-16	89.62	68.27	79.55	69	25.0×10⁶
YOLOv3	Darknet53	91.28	79.12	88.64	36	61.6×10⁶
YOLOv5s	CSPDarknet53	95.66	78.62	92.15	80	7.2×10⁶
YOLOv7	CBS+ELAN+MP+SPPCSPC	93.48	89.19	93.59	28	36.9×10⁶
本文所提算法	CBS+C3+P-SPPCSPC	95.80	94.31	95.83	68	9.3×10⁶

下载: 导出CSV

参考文献(25)

[1]	FAA. Airport foreign object debris detection equipment: AC 150/5220-24[R]. Washington. D.C.: Federal Aviation Administration, 2009: 1-13.
[2]	陈唯实, 李敬. 基于视频数据的机场跑道外来物检测[J]. 北京麻豆精品秘国产传媒学报, 2014, 40(12): 1678-1684. CHEN W S, LI J. Foreign object debris detection for airport runway with video data[J]. Journal of Beijing University of Aeronautics and Astronautics, 2014, 40(12): 1678-1684(in Chinese).
[3]	WANG Y, HUANG H, WANG J, et al. An image denoising method for arc-scanning SAR for airport runway foreign object debris detection[J]. Electronics, 2023, 12(4): 984. doi: 10.3390/electronics12040984
[4]	FUTATSUMORI S, SHIBAGAKI N. 96 GHz millimeter-wave radar system for airport surface detection purpose[C]//Proceedings of the 2022 IEEE Conference on Antenna Measurements and Applications. Piscataway: IEEE Press, 2022: 1-2.
[5]	REN S J, HAN S Y, WANG B S. Stationary and small target detection for millimeter-wave radar[C]//Proceedings of the 2022 IEEE 22nd International Conference on Communication Technology. Piscataway: IEEE Press, 2022: 1698-1702.
[6]	HAO X J, ZHAN Y H. Research on foreign object debris detection algorithm[C]//Proceedings of the 2022 International Conference on Mechanical and Electronics Engineering. Piscataway: IEEE Press, 2022: 312-318.
[7]	王国屹, 孙永荣, 张怡, 等. 背景对齐差分的机场跑道异物分块检测与跟踪算法[J]. 计算机辅助设计与图形学学报, 2021, 33(3): 413-423. WANG G Y, SUN Y R, ZHANG Y, et al. Block detection and tracking algorithm of foreign objects debris in airport runway based on background alignment and difference[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(3): 413-423(in Chinese).
[8]	ADI K, WIDODO C E, WIDODO A P, et al. Detection of foreign object debris (Fod) using convolutional neural network (CNN)[J]. Journal of Theoretical and Applied Information Technology, 2022, 100(1): 184-191.
[9]	李沙, 李春娟. 机场跑道异物检测系统设计与算法研究[J]. 现代雷达, 2021, 43(6): 80-85. LI S, LI C J. A study on design and algorithm of FOD detection and surveillance system for airport runway[J]. Modern Radar, 2021, 43(6): 80-85(in Chinese).
[10]	CAO X G, GONG G P, LIU M M, et al. Foreign object debris detection on airfield pavement using region based convolution neural network[C]//Proceedings of the 2016 International Conference on Digital Image Computing: Techniques and Applications. Piscataway: IEEE Press, 2016: 1-6.
[11]	于晨. 基于深度学习的机场跑道异物检测与识别技术研究[D]. 北京: 北京工业大学, 2019: 39-52. YU C. Research of deep learning based airport runway foreign object debris detection and recognition[D]. Beijing: Beijing University of Technology, 2019: 39-52(in Chinese).
[12]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
[13]	郭晓静, 隋昊达. 改进YOLOv3在机场跑道异物目标检测中的应用[J]. 计算机工程与应用, 2021, 57(8): 249-255. doi: 10.3778/j.issn.1002-8331.2007-0173 GUO X J, SUI H D. Application of improved YOLOv3 in foreign object debris target detection on airfield pavement[J]. Computer Engineering and Applications, 2021, 57(8): 249-255(in Chinese). doi: 10.3778/j.issn.1002-8331.2007-0173
[14]	FARHADI A, REDMON J. Yolov3: an incremental improvement[C]//Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018, 1804: 1-6.
[15]	何自芬, 陈光晨, 王森, 等. 融合自注意力特征嵌入的夜间机场跑道异物入侵检测[J]. 光学精密工程, 2022, 30(13): 1591-1605. doi: 10.37188/OPE.20223013.1591 HE Z F, CHEN G C, WANG S, et al. Detection of foreign object debris on night airport runway fusion with self-attentional feature embedding[J]. Optics and Precision Engineering, 2022, 30(13): 1591-1605(in Chinese). doi: 10.37188/OPE.20223013.1591
[16]	WU W T, LIU H, LI L L, et al. Application of local fully convolutional neural network combined with YOLO v5 algorithm in small target detection of remote sensing image[J]. PLoS One, 2021, 16(10): e0259283. doi: 10.1371/journal.pone.0259283
[17]	WANG C Y, MARK LIAO H Y, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of CNN[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE Press, 2020: 1571-1580.
[18]	VALLENDER S S. Calculation of the Wasserstein distance between probability distributions on the line[J]. Theory of Probability & Its Applications, 1974, 18(4): 784-786.
[19]	CHEN J R, KAO S H, HE H, et al. Run, don't walk: chasing higher FLOPS for faster neural networks[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2023: 12021-12031.
[20]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the Computer Vision – ECCV 2018. Berlin: Springer International Publishing, 2018: 3-19.
[21]	WANG J W, XU C, YANG W, et al. A normalized Gaussian Wasserstein distance for tiny object detection[EB/OL]. (2021-10-26)[2023-07-05]. http://doi.org/10.485501/arXiv.2110.13389.
[22]	ZHENG Z H, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 12993-13000. doi: 10.1609/aaai.v34i07.6999
[23]	杨勇, 邱根莹, 黄淑英, 等. 基于改进大气散射模型的单幅图像去雾方法[J]. 北京麻豆精品秘国产传媒学报, 2022, 48(8): 1364-1375. YANG Y, QIU G Y, HUANG S Y, et al. Single image dehazing method based on improved atmospheric scattering model[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(8): 1364-1375(in Chinese).
[24]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]//Proceedings of the Computer Vision – ECCV 2016. Berlin: Springer, 2016: 21-37.
[25]	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//Proceedings of the 2023IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2023: 7464-7475.