Object Detection in Remote Sensing Imagery with Multi-scale Deformable Convolutional Networks

doi:10.11947/j.AGCS.2018.20170595

Abstract

Abstract: Traditional target detection methods based on sliding window search paradigm and hand-craft based features are difficult to be applied to the multi-class target detection of very-high-resolution remote sensing images. In this paper,we proposed a deformable convolutional networks based multi-class target detection method by introducing deformable convolution layer and deformable RoI (Region-of-Interest) pooling layer. Specially,our method consists of two sub networks:a region proposal network aims to predict candidate regions from several layers with different filter size,and a region classification network for discrimination and regression. The quantitative comparison results on the challenging NWPU VHR-10 data set,large-scale Google Earth images, GF-2 and JL-1 images show that our method is more accurate and robust than existing algorithms.

Key words: remote sensing, object detection, deep learning, deformable convolutional layer, deformable pooling layer

CLC Number:

P237

DENG Zhipeng, SUN Hao, LEI Lin, ZHOU Shilin, ZOU Huanxin. Object Detection in Remote Sensing Imagery with Multi-scale Deformable Convolutional Networks[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(9): 1216-1227.

References

[1] CHENG Gong, HAN Junwei. A Survey on Object Detection in Optical Remote Sensing Images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 117:11-28.
[2] 胡风明, 范学花, 杨汝良,等. 利用方向性粗糙度特征对SAR图像目标检测的研究[J]. 测绘学报, 2009, 38(3):229-235. DOI:10.3321/j.issn:1001-1595.2009.03.007. HU Fengming, FAN Xuehua, YANG Ruliang, et al. Study of Target Detection of SAR Image Using Directional Roughness Feature[J]. Acta Geodaetica et Cartographica Sinica, 2009, 38(3):229-235. DOI:10.3321/j.issn:1001-1595.2009.03.007.
[3] CHENG Gong, ZHOU Peicheng, HAN Junwei. Learning Rotation-invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12):7405-7415.
[4] QIU Shaohua, WEN Gongjian, LIU Jia, et al. Unified Partial Configuration Model Framework for Fast Partially Occluded Object Detection in High-resolution Remote Sensing Images[J]. Remote Sensing, 2018, 10(3):464.
[5] QIU Shaohua, WEN Gongjian, DENG Zhipeng, et al. Automatic and Fast PCM Generation for Occluded Object Detection in High-resolution Remote Sensing Images[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10):1730-1734.
[6] QIU Shaohua, WEN Gongjian, FAN Yaxiang. Occluded Object Detection in High-resolution Remote Sensing Images using Partial Configuration Object Model[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(5):1909-1925.
[7] GIRSHICK R, DONAHUE J, DARRELL T, et al. Region-based Convolutional Networks for Accurate Object Detection and Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(1):142-158.
[8] GIRSHICK R. Fast R-CNN[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago:IEEE Press, 2015:1440-1448.
[9] 许夙晖, 慕晓冬, 赵鹏, 等. 利用多尺度特征与深度网络对遥感影像进行场景分类[J]. 测绘学报, 2016, 45(7):834-840. DOI:10.11947/j.AGCS.2016.20150623. XU Suhui, MU Xiaodong, ZHAO Peng, et al. Scene Classification of Remote Sensing Image Based on Multi-scale Feature and Deep Neural Network[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(7):834-840. DOI:10.11947/j.AGCS.2016.20150623.
[10] 何小飞, 邹峥嵘, 陶超, 等. 联合显著性和多层卷积神经网络的高分影像场景分类[J]. 测绘学报, 2016, 45(9):1073-1080. DOI:10.11947/j.AGCS.2016.20150612. HE Xiaofei, ZOU Zhengrong, TAO Chao, et al. Combined Saliency with Multi-convolutional Neural Network for High Resolution Remote Sensing Scene Classification[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(9):1073-1080. DOI:10.11947/j.AGCS.2016.20150612.
[11] 许夙晖, 慕晓冬, 张雄美, 等. 结合对抗网络与辅助任务的遥感影像无监督域适应方法[J]. 测绘学报, 2017, 46(12):1969-1977. DOI:10.11947/j.AGCS.2017.20170291. XU Suhui, MU Xiaodong, ZHANG Xiongmei, et al. Unsupervised Remote Sensing Domain Adaptation Method with Adversarial Network and Auxiliary Task[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(12):1969-1977. DOI:10.11947/j.AGCS.2017.20170291.
[12] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN:Towards Real-time Object Detection with Region Proposal Networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montréal:MIT Press, 2015:91-99.
[13] REDMON J, DIVVALA S, GIRSHICK R, et al. You Only Look Once:Unified, Real-time Object Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE, 2016:779-788.
[14] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD:Single Shot Multibox Detector[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam:Springer, 2016:21-37.
[15] DAI Jifeng, LI Yi, HE Kaiming, et al. R-Fcn:Object Detection Via Region-based Fully Convolutional Networks[C]//Advances in Neural Information Processing Systems 29. Barcelona:NIPS Press, 2016:379-387.
[16] TANG Tianyu, ZHOU Shilin, DENG Zhipeng, et al. Arbitrary-oriented Vehicle Detection in Aerial Imagery with Single Convolutional Neural Networks[J]. Remote Sensing, 2017, 9(11):1170.
[17] ZHANG Peng, NIU Xin, DOU Yong, et al. Airport Detection on Optical Satellite Images Using Deep Convolutional Neural Networks[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(8):1183-1187.
[18] YANG Yiding, ZHUANG Yin, BI Fukun, et al. M-FCN:Effective Fully Convolutional Network-based Airplane Detection Framework[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(8):1293-1297.
[19] LI Xiaobin, WANG Shengjin. Object Detection Using Convolutional Neural Networks in a Coarse-to-fine Manner[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(11):2037-2041.
[20] DENG Zhipeng, SUN Hao, ZHOU Shilin, et al. Toward Fast and Accurate Vehicle Detection in Aerial Images Using Coupled Region-based Convolutional Neural Networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(8):3652-3664.
[21] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep Residual Learning for Image Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE Press, 2016:770-778.
[22] JADERBERG M, SIMONYAN K, ZISSERMAN A. Spatial Transformer Networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems 28. Montréal:MIT Press, 2015:2017-2025.
[23] CAI Zhaowei, FAN Quanfu, FERIS R S, et al. A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam:Springer, 2016:354-370.
[24] LECUN Y, BOSER B, DENKER J S, et al. Backpropagation Applied to Handwritten Zip Code Recognition[J]. Neural Computation, 1989, 1(4):541-551.
[25] DAI Jifeng, QI Haozhi, XIONG Yuwen, et al. Deformable Convolutional Networks[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice:IEEE Press, 2017:1-13.
[26] NEUBECK A, GOOL L V. Efficient Non-maximum Suppression[C]//Proceedings of the 18th International Conference on Pattern Recognition. Hong Kong:IEEE Press, 2006:850-855.
[27] CHEN Tianqi, LI Mu, LI Yutian, et al. MXNnet:A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems[C]//Advances in Neural Information Processing Systems. Montréal:NIPS Press, 2015:1-16.