Feature-representation-transfer based road extraction method for cross-domain aerial images

doi:10.11947/j.AGCS.2020.20190274

Abstract

Abstract: Aiming at the problem of the insufficient generalization ability of traditional road extraction methods when applying to a new dataset, this paper proposes a cross-domain road extraction method that realized by feature-representation-transfer and encoder-decoder network. Firstly, a basic road extraction model based on encoder-decoder network is designed to segment the road from a single data source. Then, based on the structure of road extraction network and the principle of cycle-consistent, a cycle generative adversarial network for feature transfer of cross-domain imagery is used, which maps the feature of target city images to the domain of source data. Finally, the pre-trained road extraction model is used to segment the target domain images after the feature transfer, so that the cross-domain road extraction can be realized. The experimental results show that the proposed method improves the generalization ability of the road extraction network and can extract the road target from cross-domain images accurately and effectively. Compared with the results without feature transfer, the proposed method greatly improves the road extraction metric, and increases the F1-score by more than 50%. The proposed method does not require any annotation of the target domain images, nor does it need to fine-tune the road extraction network, while it only need to train the feature transfer model from the target domain to the source domain. Therefore, it has good application value.

Key words: road extraction, remote sensing, transfer learning, deep learning, generative adversarial network, encoder-decoder network

CLC Number:

P237

WANG Shuyang, MU Xiaodong, HE Hao, YANG Dongfang, MA Chenhui. Feature-representation-transfer based road extraction method for cross-domain aerial images[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(5): 611-621.

References

[1] 吴亮, 胡云安. 遥感图像自动道路提取方法综述[J]. 自动化学报, 2010, 36(7): 912-922. WU Liang, HU Yun’an. A survey of automatic road extraction from remote sensing images[J]. Acta Automatica Sinica, 2010, 36(7): 912-922.
[2] MNIH V, HINTON G E. Learning to detect roads in high-resolution aerial images[C]//Proceedings of European Conference on Computer Vision: Part VI, September 5-11, 2010, Heraklion, Crete, Greece. Heraklion: Springer, 210-223.
[3] MNIH V. Machine learning for aerial image labeling[D]. Toronto: University of Toronto, 2013.
[4] WANG Jun, SONG Jingwei, CHEN Mingquan, et al. Road network extraction: a neural-dynamic framework based on deep learning and a finite state machine[J]. International Journal of Remote Sensing, 2015, 36(12): 3144-3169.
[5] ALSHEHHI R, MARPU P R, WOON W L, et al. Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 130: 139-149.
[6] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, June 7-12, 2015, Boston, MA. Boston: IEEE, 2015: 3431-3440.
[7] ZHONG Zilong, LI J, CUI Weihong, et al. Fully convolutional networks for building and road extraction: preliminary results[C]//Proceedings of 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), July 10-15, 2016, Beijing, China. Beijing: IEEE, 2016: 1591-1594.
[8] WEI Yanan, WANG Zulin, XU Mai. Road structure refined CNN for road extraction in aerial image[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(5): 709-713.
[9] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer International Publishing, 2015: 234-241.
[10] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[11] 贺浩, 王仕成, 杨东方, 等. 基于Encoder-Decoder网络的遥感影像道路提取方法[J]. 测绘学报, 2019, 48(3): 330-338. DOI: 10.11947/j.AGCS.2019.20180005. HE Hao, WANG Shicheng, YANG Dongfang, et al. A road extraction method for remote sensing image based on Encoder-Decoder network[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(3): 330-338. DOI: 10.11947/j.AGCS.2019.20180005.
[12] PANBOONYUEN T, JITKAJORNWANICH K, LAWAWIROJWONG S, et al. Road segmentation of remotely-sensed images using deep convolutional neural networks with landscape metrics and conditional random fields[J]. Remote Sensing, 2017, 9(7): 9070680.
[13] SUN Tao, CHEN Zehui, YANG Wenxiang, et al. Stacked U-nets with multi-output for road extraction[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), June 18-22, 2018, Salt Lake City, UT. Salt Lake City: IEEE Computer Society, 2018.
[14] HE H, WANG S C, YANG D F, et al. Light encoder-decoder network for road extraction of remote sensing images[J]. Journal of Applied Remote Sensing. 2019, 13(3): 034510.
[15] LI Yunsheng, YUAN Lu, VASCONCELOS N. Bidirectional learning for domain adaptation of semantic segmentation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 1-8, 2019, Long Beach, USA. Long Beach: IEEE, 1-10.
[16] TSAI Y H, HUNG W C, SCHULTER S, et al. Learning to adapt structured output space for semantic segmentation[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 18-23, 2018, Salt Lake City, UT, USA. Salt Lake City: IEEE, 2018.
[17] HE Hao, YANG Dongfang, WANG Shicheng, et al. Road extraction by using atrous spatial pyramid pooling integrated encoder-decoder network and structural similarity loss[J]. Remote Sensing, 2019, 11(9): 1015.
[18] CLEVERT D A, UNTERTHINER T, HOCHREITER S. Fast and accurate deep network learning by exponential linear units (ELUs)[C]//Proceedings of the 4th International Conference on Learning Representations. San Juan, Puerto Rico: [s.n.], 2016.
[19] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]// Proceedings of the 32nd International Conference on Machine Learning, July 6-11, 2015, Lille, France. Lille: JMLR, 2015: 448-456.
[20] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation. arXiv: 1706.05587, 2017.
[21] ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of IEEE International Conference on Computer Vision, October 22-29, 2017, Venice, Italy. Venice: IEEE, 2017: 2242-2251.
[22] ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, July 21-26, 2017, Honolulu, USA. Honolulu: IEEE, 2017: 5967-5976.
[23] COSTEA D, LEORDEANU M. Aerial image geolocalization from recognition and matching of roads and intersections[C]//Proceedings of British Machine Vision Conference, September 19-22, 2016, York, UK. York: BMVA, 1-12.
[24] WIEDEMANN C, HEIPKE C, MAYER H, et al. Empirical evaluation of automatically extracted road axes[C]//Proceedings of the 9th Australasian Remote Sensing Photogrammetry Conference. Sydney: The University of New South Wales, 1998: 172-187.
[25] KINGMA D P, BA J. Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations. CA, San Diego: [s.n.], 2015.
[26] GONG Jianya. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018, 1(1): 1-15 doi:10.11947/j.JGGS.2018.0101.