Automatic Registration of Vehicle-borne Mobile Mapping Laser Point Cloud and Sequent Panoramas

doi:10.11947/j.AGCS.2018.20170520

Abstract

Abstract: An automatic registration method of mobile mapping system laser point cloud and sequence panoramic image is proposed in this paper.Firstly,hierarchical object extraction method is applied on LiDAR data to extract the building façade and outline polygons are generated to construct the skyline vectors.A virtual imaging method is proposed to solve the distortion on panoramas and corners on skylines are further detected on the virtual images combining segmentation and corner detection results.Secondly,the detected skyline vectors are taken as the registration primitives.Registration graphs are built according to the extracted skyline vector and further matched under graph edit distance minimization criteria.The matched conjugate primitives are utilized to solve the 2D-3D rough registration model to obtain the initial transformation between the sequence panoramic image coordinate system and the LiDAR point cloud coordinate system.Finally,to reduce the impact of registration primitives extraction and matching error on the registration results,the optimal transformation between the multi view stereo matching dens point cloud generated from the virtual imaging of the sequent panoramas and the LiDAR point cloud are solved by a 3D-3D ICP registration algorithm variant,thus,refine the exterior orientation parameters of panoramas indirectly.Experiments are undertaken to validate the proposed method and the results show that 1.5 pixel level registration results are achieved on the experiment dataset.The registration results can be applied to point cloud and panoramas fusion applications such as true color point cloud generation.

Key words: LiDAR point cloud, panoramas, registration, feature extraction, vehicle-borne mobile mapping system

CLC Number:

P237

CHEN Chi, YANG Bisheng, TIAN Mao, LI Jianping, ZOU Xianghong, WU Weitong, SONG Yiheng. Automatic Registration of Vehicle-borne Mobile Mapping Laser Point Cloud and Sequent Panoramas[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(2): 215-224.

References

[1] 侯艳芳, 叶泽田, 杨勇. 基于POS数据的车载面阵CCD影像与激光点云融合处理研究[J]. 遥感信息, 2011(4):76-79. HOU Yanfang, YE Zetian, YANG Yong, Research on Integrated Processing of Vehicle-borne Array CCD Images and Laser Point Cloud Based on POS Data[J]. Remote Sensing Information, 2011(4):76-79.
[2] 吴胜浩, 钟若飞. 基于移动平台的激光点云与数字影像融合方法[J]. 首都师范大学学报(自然科学版), 2011, 32(4):57-61. WU Shenghao, ZHONG Ruofei. The 3D Visualization's Approach Based on Vehicle-borne Laser Scanning[J]. Journal of Capital Normal University (Natural Science Edition), 2011, 32(4):57-61.
[3] PYLVÄNÄINEN T, ROIMELA K, VEDANTHAM R, et al. Automatic Alignment and Multi-View Segmentation of Street View Data Using 3D Shape Priors[C]//Proceedings of the 5th International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT). Paris, France:Espace Saint Martin, 2010:738-739.
[4] WANG Ruisheng, FERRIE F P, MACFARLANE J. Automatic Registration of Mobile LiDAR and Spherical Panoramas[C]//Proceedings of 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Providence, RI:IEEE, 2012:33-40.
[5] SWART A, BROERE J, VELTKAMP R, et al. Refined Non-rigid Registration of A Panoramic Image Sequence to a LiDAR Point Cloud[M]//STILLA U, ROTTENSTEINER F, MAYER H, et al. Photogrammetric Image Analysis. Berlin:Springer, 2011:73-84.
[6] CHEN C, YANG B, DENG C. Registration of the Panoramic Image Sequence onto Mobile Laser Scanning Point Cloud Using Linear Features[C]//The International Symposium on Mobile Mapping Technology (MMT2013). Taiwan, China:2013:1-9.
[7] TANEJA A, BALLAN L, POLLEFEYS M. Geometric Change Detection in Urban Environments Using Images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(11):2193-2206.
[8] HOFMANN S, EGGERT D, BRENNER C. Skyline Matching Based Camera Orientation from Images and Mobile Mapping Point Clouds[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2014, Ⅱ-5:181-188.
[9] VIOLA P, WELLS Ⅲ W M. Alignment by Maximization of Mutual Information[J]. International Journal of Computer Vision, 1997, 24(2):137-154.
[10] NELDER J A, MEAD R. A Simplex Method for Function Minimization[J]. The Computer Journal, 1965, 7(4):308-313.
[11] RUSINKIEWICZ S, LEVOY M. Efficient Variants of the ICP Algorithm[C]//Proceedings of the 3rd International Conference on 3-D Digital Imaging and Modeling. Quebec City, Canada:IEEE, 2001:145-152.
[12] YANG Bisheng, DONG Zhen, ZHAO Gang, et al. Hierarchical Extraction of Urban Objects from Mobile Laser Scanning Data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 99:45-57.
[13] ROTHER C, KOLMOGOROV V, BLAKE A. "GrabCut":Interactive Foreground Extraction Using Iterated Graph Cuts[C]//Proceedings of ACM SIGGRAPH 2004. New York, NY:ACM, 2004:309-314.
[14] AWRANGJEB M, LU Guojun, FRASER C S. Performance Comparisons of Contour-based Corner Detectors[J]. IEEE Transactions on Image Processing, 2012, 21(9):4167-4179.
[15] CHRISTY S, HORAUD R. Iterative Pose Computation from Line Correspondences[J]. Computer Vision and Image Understanding, 1999, 73(1):137-144.
[16] 邓非, 张祖勋, 张剑清. 利用激光扫描和数码相机进行古建筑三维重建研究[J]. 测绘科学, 2007, 32(2):29-30. DENG Fei, ZHANG Zuxun, ZHANG Jianqing. 3D Reconstruction of Old Architecture by Laser Scanner and Digital Camera[J]. Science of Surveying and Mapping, 2007, 32(2):29-30.
[17] 王晏民, 胡春梅. 一种地面激光雷达点云与纹理影像稳健配准方法[J]. 测绘学报, 2012, 41(2):266-272. WANG Yanmin, HU Chunmei. A Robust Registration Method for Terrestrial LiDAR Point Clouds and Texture Image[J]. Acta Geodaetica et Cartographica Sinica, 2012, 41(2):266-272.
[18] LIU L, STAMOS I. Automatic 3D to 2D Registration for the Photorealistic Rendering of Urban Scenes[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego:IEEE, 2005:137-143.
[19] RAMALINGAM S, BOUAZIZ S, STURM P, et al. SKYLINE2GPS:Localization in Urban Canyons Using Omni-skylines[C]//Proceedings of 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Taipei, Taiwan, China:IEEE, 2010:3816-3823.
[20] SATTLER T, LEIBE B, KOBBELT L. Fast Image-based Localization Using Direct 2D-to-3D Matching[C]//Proceedings of 2011 IEEE International Conference on Computer Vision (ICCV). Barcelona, Spain:IEEE, 2011:667-674.
[21] KNOPP J, SIVIC J, PAJDLA T. Avoiding Confusing Features in Place Recognition[M]//DANⅡLIDIS K, MARAGOS P, PARAGIOS N. Computer Vision-ECCV 2010. Berlin:Springer, 2010:748-761.
[22] LI Zhuqiang, ZHANG Liqiang, ZHONG Ruofei, et al. Classification of Urban Point Clouds:A Robust Supervised Approach with Automatically Generating Training Data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(3):1207-1220.
[23] KWAK E. Automatic 3D Building Model Generation by Integrating LiDAR and Aerial Images Using a Hybrid Approach[D]. Calgary:University of Calgary, 2013.
[24] AWRANGJEB M, LU Guojun, FRASER C S, et al. A Fast Corner Detector Based on the Chord-to-point Distance Accumulation Technique[C]//Proceedings of 2009 Digital Image Computing:Techniques and Applications. Melbourne, VIC, Australia:IEEE, 2009:519-525.
[25] 袁修孝, 张雪萍, 付建红. 高斯-克吕格投影坐标系下POS角元素的转换方法[J]. 测绘学报, 2011, 40(3):338-344. YUAN Xiuxiao, ZHANG Xueping, FU Jianhong. Transformation of Angular Elements Obtained Via A Position and Orientation System in Gauss-Kruger Projection Coordinate System[J]. Acta Geodaetica et Cartographica Sinica, 2011, 40(3):338-344.
[26] ZHOU Kun, HOU Qinming, WANG Rui, et al. Real-Time KD-Tree Construction on Graphics Hardware[J]. ACM Transactions on Graphics (TOG), 2008, 27(5):126.
[27] SNAVELY N, SEITZ S M, SZELISKI R. Modeling the World from Internet Photo Collections[J]. International Journal of Computer Vision, 2008, 80(2):189-210.
[28] TOLA E, LEPETIT V, FUA P. DAISY:An Efficient Dense Descriptor Applied to Wide-baseline Stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(5):815-830.
[29] YANG Bisheng, CHEN Chi. Automatic Registration of UAV-borne Sequent Images and LiDAR Data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 101:262-274.
[30] CHEN Y, MEDIONI G. Object Modeling by Registration of Multiple Range Images[C]//Proceedings of 1991 IEEE International Conference on Robotics and Automation. Sacramento, CA:IEEE, 1991:2724-2729.
[31] POMERLEAU F, COLAS F, FERLAND F, et al. Relative Motion Threshold for Rejection in ICP Registration[M]//HOWARD A, IAGNEMMA K, KELLY A. Field and Service Robotics. Berlin:Springer, 2010:229-238.