Generalized photogrammetry of spaceborne, airborne and terrestrial multi-source remote sensing datasets

doi:10.11947/j.AGCS.2021.20200245

Abstract

Abstract: Since the 21st century, with the rapid development of cloud computing, big data, internet of things, machine learning and other information technology fields, human beings have entered a new era of artificial intelligence. The subject of photogrammetry has also followed the tide of the new round of scientific and technological revolution and developed rapidly into the brand-new generalized photogrammetry and entered the era of integrated intelligent photogrammetry. Its carrier platform, instruments and data processing theories as well as application fields have also changed significantly. The multi-sensor and multi-level integrated stereo observation technologies from spaceborne, airborne and terrestrial platforms have been greatly developed. In this paper, the novel concept of generalized photogrammetry is first put forward, and its subject connotation, development characteristics and some key technologies and applications are discussed in details. Under the brand-new generalized photogrammetry framework, data acquisition presents the characteristics of multi-angle imaging, multi-modal collaboration, multi-time integration, multi-scale linkage, while data processing presents the trends of multi-feature coupling, multi-control constraints, multi architecture processing, and multi-disciplinary intersection. The all-round development and intelligent service of the general photogrammetry still need to make greater breakthroughs in the aspects of spaceborne, airborne and terrestrial multi perspective or multi-modal image processing, intelligent information extraction and monitoring, combined 3D modeling with point cloud and image, autonomous control of unmanned system, visual inspection of intelligent manufacturing system, etc. Finally, new theories and technologies from real-time or quasi real-time intelligent geometric processing of multi-source remote sensing datasets to information extraction and intelligent service need to be established, which will make a well foundation to meet the new eara of intelligent surveying and mapping.

Key words: generalized photogrammetry, integrated spaceborne, airborne and terrestrial observation, multi-sensor integration, multi-source remote sensing data, intelligent photogrammetry

CLC Number:

ZHANG Yongjun, ZHANG Zuxun, GONG Jianya. Generalized photogrammetry of spaceborne, airborne and terrestrial multi-source remote sensing datasets[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(1): 1-11.

References

[1] 龚健雅, 季顺平. 从摄影测量到计算机视觉[J]. 武汉大学学报(信息科学版), 2017, 42(11):1518-1522, 1615. GONG Jianya, JI Shunping. From photogrammetry to computer vision[J]. Geomatics and Information Science of Wuhan University, 2017, 42(11):1518-1522, 1615.
[2] SEFERCIK U G, ALKAN M, BUYUKSALIH G, et al. Generation and validation of high-resolution DEMs from Worldview-2 stereo data[J]. The Photogrammetric Record, 2013, 28(144):362-374.
[3] ZHANG Yongjun, XIONG Jinxin, HAO Lijuan. Photogrammetric processing of low-altitude images acquired by unpiloted aerial vehicles[J]. The Photogrammetric Record, 2011, 26(134):190-211.
[4] COLOMINA I, MOLINA P. Unmanned aerial systems for photogrammetry and remote sensing:a review[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 92:79-97.
[5] ZHANG Yongjun, ZHANG Zuxun, ZHANG Jianqing. Automatic measurement of industrial sheetmetal parts with CAD data and non-metric image sequence[J]. Computer Vision and Image Understanding, 2006, 102(1):52-59.
[6] LUHMANN T. Close range photogrammetry for industrial applications[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2010, 65(6):558-569.
[7] FRASER C. Automatic camera calibration in close range photogrammetry[J]. Photogrammetric Engineering and Remote Sensing, 2013, 79(4):381-388.
[8] REINARTZ P, MVLLER R, SCHWIND P, et al. Orthorectification of VHR optical satellite data exploiting the geometric accuracy of TerraSAR-X data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2011, 66(1):124-132.
[9] ZHANG Yongjun, XIONG Xiaodong, ZHENG Maoteng, et al. LiDAR strip adjustment using multifeatures matched with aerial images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(2):976-987.
[10] CHEN Gang, METZ M R, RIZZO D M, et al. Object-based assessment of burn severity in diseased forests using high-spatial and high-spectral resolution MASTER airborne imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 102:38-47.
[11] RIZZOLI P, MARTONE M, GONZALEZ C, et al. Generation and performance assessment of the global TanDEM-X digital elevation model[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 132:119-139.
[12] WARRICK J A, RITCHIE A C, ADELMAN G, et al. New techniques to measure cliff change from historical oblique aerial photographs and structure-from-motion photogrammetry[J]. Journal of Coastal Research, 2017, 33(1):39-55.
[13] ZHANG Xujie, ZHAO Pengcheng, HU Qingwu, et al. A UAV-based panoramic oblique photogrammetry (POP) approach using spherical projection[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159:198-219.
[14] 张永军, 刘经南, 张祖勋, 等. 基于非量测CCD摄像机的钣金件误差检测[J]. 测绘学报, 2004, 33(2):132-137. DOI:10.3321/j.issn:1001-1595.2004.02.008. ZHANG Yongjun, LIU Jingnan, ZHANG Zuxun, et al. Imprecision inspection of sheetmetal parts with non-metric CCD camera[J]. Acta Geodaetica et Cartographica Sinica, 2004, 33(2):132-137. DOI:10.3321/j.issn:1001-1595.2004.02.008.
[15] 张祖勋, 张剑清. 广义点摄影测量及其应用[J]. 武汉大学学报(信息科学版), 2005, 30(1):1-5. ZHANG Zuxun, ZHANG Jianqing. Generalized point photogrammetry and its application[J]. Geomatics and Information Science of Wuhan University, 2005, 30(1):1-5.
[16] ZHANG Zuxun, ZHANG Yongjun, ZHANG Jianqing, et al. Photogrammetric modeling of linear features with generalized point photogrammetry[J]. Photogrammetric Engineering & Remote Sensing, 2008, 74(9):1119-1127.
[17] 张祖勋. 从数字摄影测量工作站(DPW)到数字摄影测量网格(DPGrid)[J]. 武汉大学学报(信息科学版), 2007, 32(7):565-571. ZHANG Zuxun. From digital photogrammetry workstation (DPW) to digital photogrammetry grid (DPGrid)[J]. Geomatics and Information Science of Wuhan University, 2007, 32(7):565-571.
[18] ZHANG Yongjun, HU Binghua, ZHANG Jianqing. Relative orientation based on multi-features[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2011, 66(5):700-707.
[19] 张祖勋, 陶鹏杰. 谈大数据时代的"云控制"摄影测量[J]. 测绘学报, 2017, 46(10):1238-1248. DOI:10.11947/j.AGCS.2017.20170337. ZHANG Zuxun, TAO Pengjie. An overview on "cloud control" photogrammetry in big data era[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1238-1248. DOI:10.11947/j.AGCS.2017.20170337.
[20] ZHANG Yongjun, WANG Bo, ZHANG Zuxun, et al. Fully automatic generation of geoinformation products with Chinese ZY-3 satellite imagery[J]. The Photogrammetric Record, 2014, 29(148):383-401.
[21] ZHANG Yongjun, WAN Yi, HUANG Xinhui, et al. DEM-assisted RFM block adjustment of pushbroom nadir viewing HRS imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(2):1025-1034.
[22] VASSILAKI D I, STAMOS A A. TanDEM-X DEM:comparative performance review employing LiDAR data and DSMs[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 160:33-50.
[23] CHEN Jun, DOWMAN I, LI Songnian, et al. Information from imagery:ISPRS scientific vision and research agenda[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 115:3-21.
[24] 张祖勋. 由数字摄影测量的发展谈信息化测绘[J]. 武汉大学学报(信息科学版), 2008, 33(2):111-115. ZHANG Zuxun. On informatization of surveying and mapping from the development of digital photogrammetry[J]. Geomatics and Information Science of Wuhan University, 2008, 33(2):111-115.
[25] 季顺平. 智能摄影测量学导论[M]. 北京:科学出版社, 2018:172. JI Shunping. An introduction to intelligent photogrammetry[M]. Beijing:Science Press, 2018:172.
[26] LI Yansheng, ZHANG Yongjun, HUANG Xin, et al. Deep networks under scene-level supervision for multi-class geospatial object detection from remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 146:182-196.
[27] PENG Daifeng, ZHANG Yongjun, GUAN Haiyan. End-to-end change detection for high resolution satellite images using improved UNet++[J]. Remote Sensing, 2019, 11:1382. DOI:10.3390/rs11111382.
[28] LIU Xinyi, ZHANG Yongjun, LING Xiao, et al. TopoLAP:topology recovery for building reconstruction by deducing the relationships between linear and planar primitives[J]. Remote Sensing, 2019, 11(11):1372. DOI:10.3390/rs11111372.
[29] STATHAM N. Use of photogrammetry in video games:a historical overview[J]. Games and Culture, 2020, 15(3):289-307.
[30] WANG Meng, DONG Han, ZAHNG Wei, et al. An end-to-end auto-driving method based on 3D LiDAR[J]. Journal of Physics:Conference Series, 2019, 1288:012061.
[31] LIU Dongjie, ZHAO Jin, XI Axin, et al. Data augmentation technology driven by image style transfer in self-driving car based on end-to-end Learning[J]. CMES-Computer Modeling in Engineering & Sciences, 2020, 122(2):593-617.
[32] LI Rongxing, HWANGBO J, CHEN Yunhang, et al. Rigorous photogrammetric processing of HiRISE stereo imagery for mars topographic mapping[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(7):2558-2572.
[33] 邸凯昌, 刘斌, 辛鑫, 等. 月球轨道器影像摄影测量制图进展及应用[J]. 测绘学报, 2019, 48(12):1562-1574. DOI:10.11947/j.AGCS.2019.20190462. DI Kaichang, LIU Bin, XIN Xin, et al. Advances and applications of lunar photogrammetric mapping using orbital images[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1562-1574. DOI:10.11947/j.AGCS.2019.20190462.
[34] HE Yuan, ZHENG Shunyi, ZHU Fengbo, et al. Real-time 3D reconstruction of thin surface based on laser line scanner[J]. Sensors, 2020, 20(2):534. DOI:10.3390/s20020534.
[35] ZHANG Zuxun, ZHANG Yongjun, KE Tao, et al. Photogrammetry for first response in Wenchuan earthquake[J]. Photogrammetric Engineering and Remote Sensing, 2009, 75(5):510-513.
[36] SCHUMANN G J P, BRAKENRIDGE G R, KETTNER A J, et al. Assisting flood disaster response with earth observation data and products:a critical assessment[J]. Remote Sensing, 2018, 10(8):1230. DOI:10.3390/rs10081230.
[37] SCHENK T. Digital photogrammetry[M]. Laurelville:Terra-Science, 1999.
[38] 李德仁. 展望大数据时代的地球空间信息学[J]. 测绘学报, 2016, 45(4):379-384. DOI:10.11947/j.AGCS.2016.20160057. LI Deren. Towards geo-spatial information science in big data era[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(4):379-384. DOI:10.11947/j.AGCS.2016.20160057.
[39] 李德仁. 从测绘学到地球空间信息智能服务科学[J]. 测绘学报, 2017, 46(10):1207-1212. DOI:10.11947/j.AGCS.2017.20170263. LI Deren. From geomatics to geospatial intelligent service science[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1207-1212. DOI:10.11947/j.AGCS.2017.20170263.
[40] 刘经南, 郭文飞, 郭迟, 等. 智能时代泛在测绘的再思考[J]. 测绘学报, 2020, 49(4):403-414. DOI:10.11947/j.AGCS.2020.20190539. LIU Jingnan, GUO Wenfei, GUO Chi, et al. Rethinking ubiquitous mapping in the intelligent age[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(4):403-414. DOI:10.11947/j.AGCS.2020.20190539.
[41] 王艳东, 龚健雅. 空间信息智能服务理论与方法[M]. 北京:科学出版社, 2012. WANG Yandong, GONG Jianya. Theory and method of spatial information intelligent service[M]. Beijing:Science Press, 2012.
[42] 龚健雅, 王密, 杨博. 高分辨率光学卫星遥感影像高精度无地面控制精确处理的理论与方法[J]. 测绘学报, 2017, 46(10):1255-1261. DOI:10.11947/j.AGCS.2017.20170307. GONG Jianya, WANG Mi, YANG Bo. High-precision geometric processing theory and method of high-resolution optical remote sensing satellite imagery without GCP[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1255-1261. DOI:10.11947/j.AGCS.2017.20170307.
[43] 王晋, 张勇, 张祖勋, 等. ICESat激光高程点辅助的天绘一号卫星影像立体区域网平差[J]. 测绘学报, 2018, 47(3):359-369. DOI:10.11947/j.AGCS.2018.20170425. WANG Jin, ZHANG Yong, ZHANG Zuxun, et al. ICESat laser points assisted block adjustment for mapping satellite-1 stereo imagery[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(3):359-369. DOI:10.11947/j.AGCS.2018.20170425.
[44] SUN Yanbiao, SUN Huabo, YAN Lei, et al. RBA:reduced bundle adjustment for oblique aerial photogrammetry[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 121:128-142.
[45] ZHANG Yongjun, LIN Liwen, ZHENG Maoteng, et al. Combined bundle block adjustment with spaceborne linear array and airborne frame array imagery[J]. The Photogrammetric Record, 2013, 28(142):162-177.
[46] 陈锐志, 王磊, 李德仁, 等. 导航与遥感技术融合综述[J]. 测绘学报, 2019, 48(12):1507-1522. DOI:10.11947/j.AGCS.2019.20190446. CHEN Ruizhi, WANG Lei, LI Deren, et al. A survey on the fusion of the navigation and the remote sensing techniques[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1507-1522. DOI:10.11947/j.AGCS.2019.20190446.
[47] 李德仁. 展望5G/6G时代的地球空间信息技术[J]. 测绘学报, 2019, 48(12):1475-1481. DOI:10.11947/j.AGCS.2019.20190437. LI Deren. Towards geospatial information technology in 5G/6G era[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1475-1481. DOI:10.11947/j.AGCS.2019.20190437.
[48] 杨元喜. 弹性PNT基本框架[J]. 测绘学报, 2018, 47(7):893-898. DOI:10.11947/j.AGCS.2018.20180149. YANG Yuanxi. Resilient PNT concept frame[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(7):893-898. DOI:10.11947/j.AGCS.2018.20180149.
[49] 唐新明, 陈继溢, 李国元, 等. 资源三号02星激光测高误差分析与指向角粗标定[J]. 武汉大学学报(信息科学版), 2018, 43(11):1611-1619. TANG Xinming, CHEN Jiyi, LI Guoyuan, et al. Error analysis and preliminary pointing angle calibration of laser altimeter on Ziyuan-302 satellite[J]. Geomatics and Information Science of Wuhan University, 2018, 43(11):1611-1619.
[50] KWONG I H Y, FUNG T. Tree height mapping and crown delineation using LiDAR, large format aerial photographs, and unmanned aerial vehicle photogrammetry in subtropical urban forest[J]. International Journal of Remote Sensing, 2020, 41(14):5228-5256.
[51] 冯文卿, 张永军. 利用多尺度融合进行面向对象的遥感影像变化检测[J]. 测绘学报, 2015, 44(10):1142-1151. DOI:10.11947/j.AGCS.2015.20140260. FENG Wenqing, ZHANG Yongjun. Object-oriented change detection for remote sensing images based on multi-scale fusion[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(10):1142-1151. DOI:10.11947/j.AGCS.2015.20140260.
[52] LI Hongliang, LI Dong, LI Yunhua. A multi-index assessment method for evaluating coverage effectiveness of remote sensing satellite[J]. Chinese Journal of Aeronautics, 2018, 31(10):2023-2033.
[53] ZHANG Yongjun, ZHANG Zuxun, ZHANG Jianqing, et al. 3D building modelling with digital map, LiDAR data and video image sequences[J]. The Photogrammetric Record, 2005, 20(111):285-302.
[54] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[55] XIE Xunwei, ZHANG Yongjun, LING Xiao, et al. A novel extended phase correlation algorithm based on log-Gabor filtering for multimodal remote sensing image registration[J]. International Journal of Remote Sensing, 2019, 40(14):5429-5453.
[56] YAN Xiaohu, ZHANG Yongjun, ZHANG Dejun, et al. Multimodal image registration using histogram of oriented gradient distance and data-driven grey wolf optimizer[J]. Neurocomputing, 2020, 392:108-120.
[57] MUR-ARTAL R, TARDÓS J D. ORB-SLAM2:an open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[58] 张永军, 王博, 黄旭, 等. 影像匹配粗差的局部矢量面元剔除方法[J]. 测绘学报, 2014, 43(7):717-723. DOI:10.13485/j.cnki.11-2089.2014.0092. ZHANG Yongjun, WANG Bo, HUANG Xu, et al. Eliminating of image matching gross errors based on local vector field[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(7):717-723. DOI:10.13485/j.cnki.11-2089.2014.0092.
[59] ZHENG Maoteng, ZHOU Shunping, XIONG Xiaodong, et al. A new GPU bundle adjustment method for large-scale data[J]. Photogrammetric Engineering & Remote Sensing, 2017, 83(9):633-641.
[60] 龚健雅, 钟燕飞. 光学遥感影像智能化处理研究进展[J]. 遥感学报, 2016, 20(5):733-747. GONG Jianya, ZHONG Yanfei. Survey of intelligent optical remote sensing image processing[J]. Journal of Remote Sensing, 2016, 20(5):733-747.
[61] 龚健雅, 季顺平. 摄影测量与深度学习[J]. 测绘学报, 2018, 47(6):693-704. DOI:10.11947/j.AGCS.2018.20170640. GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):693-704. DOI:10.11947/j.AGCS.2018.20170640.
[62] 郑太雄, 黄帅, 李永福, 等. 基于视觉的三维重建关键技术研究综述[J]. 自动化学报, 2020, 46(4):631-652. ZHENG Taixiong, HUANG Shuai, LI Yongfu, et al. Key techniques for vision based 3D reconstruction:a review[J]. Acta Automatica Sinica, 2020, 46(4):631-652.
[63] 王广帅, 万一, 张永军. 交叉点结构特征约束的机载LiDAR点云与多视角航空影像配准[J]. 地球信息科学学报, 2020, 22(9):1868-1877. WANG Guangshuai, WAN Yi, ZHANG Yongjun. Registration of airborne LiDAR data and multi-view aerial images constrained by junction structure features[J]. Journal of Geo-information Science, 2020, 22(9):1868-1877.
[64] 徐德民. 智能无人系统发展现状及趋势[R]. 西安:西北工业大学, 2019. XU Demin. Progresses and prospects of intelligent unmanned systems[R]. Xi'an:Northwesterl Polytechnical University, 2019.
[65] 王耀南, 陈铁健, 贺振东, 等. 智能制造装备视觉检测控制方法综述[J]. 控制理论与应用, 2015, 32(3):273-286. WANG Yaonan, CHEN Tiejian, HE Zhendong, et al. Review on the machine vision measurement and control technology for intelligent manufacturing equipment[J]. Control Theory & Applications, 2015, 32(3):273-286.
[66] CZIMMERMANN T, CIUTI G, MILAZZO M, et al. Visual-based defect detection and classification approaches for industrial applications:a survey[J]. Sensors, 2020, 20(5):1459. DOI:10.3390/s20051459.
[67] NEOGI N, MOHANTA D K, DUTTA P K. Review of vision-based steel surface inspection systems[J]. EURASIP Journal on Image and Video Processing, 2014, 2014:50. DOI:10.1186/1687-5281-2014-50.
[68] CHA Y J, CHOI W, SUH G, et al. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types[J]. Computer-aided Civil and Infrastructure Engineering, 2018, 33(9):731-747.