[1]李文波,王 玉,王明泉,等.瞬态场景下基于光流法的红外视频插帧算法研究[J].机械与电子,2024,42(04):15-21.
　LI Wenbo,WANG Yu,WANG Mingquan,et al.Research on Infrared Video Frame Interpolation Algorithm Based on Optical Flow Method in Transient Scene[J].Machinery & Electronics,2024,42(04):15-21.
点击复制

瞬态场景下基于光流法的红外视频插帧算法研究()
分享到：

《机械与电子》[ISSN:1001-2257/CN:52-1052/TH]

卷:
42
期数:
2024年04期
页码:
15-21
栏目:
研究与设计
出版日期:
2024-04-23

文章信息/Info

Title:
Research on Infrared Video Frame Interpolation Algorithm Based on Optical Flow Method in Transient Scene
文章编号:
1001-2257 ( 2024 ) 04-0015-07
作者:
李文波; 王 玉; 王明泉; 商奥雪; 丰晓钰
中北大学信息与通信工程学院,山西 太原 030051
Author(s):
LI Wenbo ;  WANG Yu ;  WANG Mingquan ;  SHANG Aoxue ;  FENG Xiaoyu
( School of Information and Communication Engineering , North University of China , Taiyuan 030051 , China )
关键词:
红外视频插帧; 注意力机制; 光流; 特征融合
Keywords:
infrared video frame interpolation ;  attention mechanism ;  optical flow ;  feature fusion
分类号:
TP391.4
文献标志码:
A
摘要:
针对现有红外图像插帧方法,在瞬态场景下均不能得到鲁棒性较好的插入帧红外图像,提出了一种基于注意力的多尺度、多分支光流网络,提取相邻 2 帧红外图像光流信息,每个分支分别学习一种光流信息,利用多尺度特征融合模块在每个尺度上聚焦局部重要信息。设计了一个多光流特征重加权模块,根据通道注意力自适应地选择每个光流的特征。经实验结果证明,所提方法可以很好地完成插帧任务,其性能与最先进的方法相比较更具有优越性。
Abstract:
According to the existing infrared image intercalation methods , the infrared image with good robustness can not be obtained in the transient scene.A multi-scale and multi-branch optical flow network based on attention is proposed.Optical flow information of two adjacent infrared images is extracted , and each branch learns one optical flow information respectively.Then , multi-scale feature fusion module is used to focus on locally important information at each scale.A multi-optical flow feature reweighting module is designed to select the features of each optical flow adaptively according to channel attention.The experimental results show that the proposed method can complete the frame interpolation task well , and its performance is more superior than that of the state of the art algorithms.

参考文献/References:

[ 1 ] RING E F J , AMMER K.Infrared thermal imaging in medicine [ J ] .Physiological measurement , 2012 , 33 ( 3 ):33-46.

[ 2 ] JIANG H Z , SUN D Q , JAMPANI V , et al.Super slomo : High quality estimation of multiple intermediate frames for video interpolation [ C ] ∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018 : 9000-9008.
[ 3 ] JONSCHKOWSKI R , STONE A , BARRON J T , et al.What matters in unsupervised optical flow [ C ] ∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2020 : 557-572.
[ 4 ] ILG E , MAYER N , SAIKIA T , et al.Flownet 2.0 : evolution of optical flow estimation with deep networks[ C ] ∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017 : 2462-2470.
[ 5 ] SUN D , YANG X , LIU M Y , et al.Pwc-net : CNNs for optical flow using pyramid , warping , and cost volume [ C ] ∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018 : 8934-8943.
[ 6 ] 董成威 . 基于深度学习的视频插帧算法研究[ D ] . 厦门:厦门大学,2020.
[ 7 ] SZEGEDY C , LIU W , JIA Y , et al.Going deeper with convolutions [ C ] ∥Proceedings of the IEEE Conference on computer Vision and Pattern Recognition , 2015 : 1-9.
[ 8 ] SI C , YU W , ZHOU P , et al.Inception transformer [ J ] .Advances in neural information processing systems , 2022 , 35 : 23495-23509.
[ 9 ] CHEN Z , HE T , JIN X , et al.Learning for video compression [ J ] .IEEE Transactions on circuits and systems for video technology , 2019 , 30 ( 2 ): 566-576.
[ 10 ] 福禄克红外热像仪“让你看见温度”[ J ] . 设备管理与维修,2017 ( 2 ): 8.
[ 11 ] 中国传媒大学 . 一 种红外热像仪帧率扩展装置:201820660138.8 [ P ] .2018-12-11.
[ 12 ] SANNA A , LAMBERTI F.Advances in target detection and tracking in forward-looking infraRed ( FLIR ) imagery [ J ] .Sensors , 2014 , 14 ( 11 ):20297-20303.
[ 13 ] BAO W , LAI W S , ZHANG X , et al.Memc-net : motion estimation and motion compensation driven neural network for video interpolation and enhancement[ J ] .IEEE Transactions on pattern analysis and machine intelligence , 2019 , 43 ( 3 ): 933-948.
[ 14 ] CHOI G , HEO P G , PARK H W.Triple-frame based bi-directional motion estimation for motion compensated frame interpolation [ J ] .IEEE Transactions on circuits and systems for video technology ,2019 , 29 ( 5 ): 1251-1258.
[ 15 ] DOSOVITSKIY A , FISCHER P , ILG E , et al.Flownet : learning optical flow with convolutional networks[ C ] ∥Proceedings of the IEEE International Conference on Computer Vision , 2015 : 2758-2766.
[16 ] NIKLAUS S , LIU F.Context-aware synthesis for video frame interpolation [ C ] ∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018 : 1701-1710.
[ 17 ] CHOI M , KIM H , HAN B , et al.Channel attention is all you need for video frame interpolation [ C ] ∥Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 7 ): 10663-10671.
[ 18 ] ZHANG J H , LIU F , ZHOU A M.Off-TANet : A lightweight neural micro-expression recognizer with optical flow features and integrated attention mechanism [ C ] ∥Pacific Rim International Conference on Artificial Intelligence , 2021 : 266-279.
[ 19 ] ZHANG C , GE L , CHEN Z , et al.Refined TV-L 1 optical flow estimation using joint filtering [ J ] .IEEE Transactions on multimedia , 2019 , 22 ( 2 ): 349-364.
[ 20 ] TEUTSCH M , MULLER T , HUBER M , et al.Low resolution person detection with a moving thermal infrared camera by hot spot classification [ C ] ∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops , 2014 : 209-216.
[ 21 ] GUAN L.Weight prediction boosts the convergence of adamW [ C ] ∥Pacific-Asia Conference on Knowledge Discovery and Data Mining , 2023 : 329-340.
[ 22 ] NIKLAUS S , MAI L , LIU F.Video frame interpolation via adaptive separable convolution [ C ] ∥Proceedings of the IEEE International Conference on Computer Vision , 2017 : 261-270.
[ 23 ] ZHANG R , ISOLA P , EFROS A A , et al.The unreasonable effectiveness of deep features as a perceptual metric [ C ] ∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018 : 586-595.
[ 24 ] LIU Z , YEH R A , TANG X , et al.Video frame synthesis using deep voxel flow [ C ] ∥Proceedings of the IEEE International Conference on Computer Vision , 2017 : 4463-4471.
[ 25 ] XU X Y , LI S Y , SUN W X , et al.Quadratic video interpolation [ C ] ∥ Proceedings of the IEEE International Conference on Computer Vision , 2020 :1636-1645.
[ 26 ] SIM H , OH J , KIM M.Xvfi : extreme video frame interpolation [ C ] ∥Proceedings of the IEEE / CVF International Conference on Computer Vision , 2021 :14489-14498.
[ 27 ] 马境远,王川铭 . 一种多尺度光流预测与融合的实时视频插帧方 法[ J ] . 小 型 微型 计 算 机 系 统,2021 , 42( 12 ): 2567-2571.

相似文献/References:

[1]杨永灿,刘 韬,柳小勤,等.基于注意力机制的一维卷积神经网络行星齿轮箱故障诊断[J].机械与电子,2021,(10):3.
　YANG Yongcan,LIU Tao,LIU Xiaoqin,et al.Fault Diagnosis of Planetary Gearbox Based on One-dimensional Convolutional Neural Network with Attention Mechanism[J].Machinery & Electronics,2021,(04):3.
[2]王 青,吕绪山,党 帅,等.基于深度学习的纱管识别方法研究[J].机械与电子,2023,41(12):20.
　WANG Qing,LYU Xushan,DANG Shuai,et al.Research on Yarn Bobbin Detection Method Based on Deep Learning[J].Machinery & Electronics,2023,41(04):20.
[3]王 凡,甄子洋,邓 敏.基于 CNN-GRU-Attention 的道岔故障诊断算法研究[J].机械与电子,2024,42(06):10.
　WANG Fan,ZHEN Ziyang,DENG Min.Research on Turnout Fault Diagnosis Algorithm Based on CNN-GRU-Attention[J].Machinery & Electronics,2024,42(04):10.
[4]鲍克鹏,许 涛,贾雪磊,等.基于改进双边分割网络的视觉测振方法[J].机械与电子,2025,(03):29.
　BAO Kepeng,XU Tao,JIA Xuelei,et al.Visual Vibration Measurement Method Based on an Improved Bilateral Segmentation Network[J].Machinery & Electronics,2025,(04):29.
[5]苑 超,丁晓森,朱 雷,等.基于改进 VMD 与 Transformer 的 GIS 局部放电信号识别[J].机械与电子,2025,(04):27.
　YUAN Chao,DING Xiaosen,ZHU Lei,et al.GIS Partial Discharge Signal Recognition Based on Improved VMD and Transformer[J].Machinery & Electronics,2025,(04):27.

备注/Memo

备注/Memo:
收稿日期: 2023-09-15
基金项目:山西省重点研发计划( 201803D121069 );山西省高等学校科技创新项目( 2020L0624 )
作者简介:李文波 ( 1997- ),男,河南卫辉人,硕士研究生,研究方向为图像处理;王 玉 ( 1979- ),女,山西太原人,博士,副教授,硕士研究生导师,研究方向为医学图像配准和融合、多维信号处理、三维可视化、荧光分子成像、工业无损检测;王明泉 ( 1970- ),山西朔州人,男,博士,教授,博士研究生导师,研究方向为图像处理,工业检测与识别;商奥雪 ( 1999- ),女,辽宁朝阳人,硕士研究生,研究方向为头颈部医学图像配准;丰晓钰 ( 1998- ),女,山西朔州人,硕士研究生,研究方向为图像分割。
更新日期/Last Update: 2024-04-29