A vision-aided geo-registration method for outdoor ARGIS in urban environments based on 2D maps

doi:10.11947/j.AGCS.2019.20190007

Abstract

Abstract: The accuracy of the outdoor 6DOF absolute pose obtained by the current portable pose sensor is usually insufficient in urban environments, which makes the geo-registration accuracy of outdoor AR poorly. Aiming at this issue, a method and technical framework for outdoor vision-aided localization is proposed with 2D maps to improve the outdoor ARGIS geo-registration accuracy. Based on the initial pose obtained by the pose sensor, the basic principles of image localization and pose optimization based on 2D maps are expounded in detail. The experimental results show that the proposed method can effectively optimize the initial pose obtained by the pose sensor, and thus improve the geo-registration accuracy of outdoor AR.

Key words: outdoor augmented reality, ARGIS, geo-registration, vision-aided registration, hybrid registration, position-posture sensor, evaluation of registration accuracy, geo-localization

CLC Number:

P208

DENG Chen, YOU Xiong, ZHANG Weiwei, ZHI Meixia. A vision-aided geo-registration method for outdoor ARGIS in urban environments based on 2D maps[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(10): 1305-1319.

References 33

[1]	杜清运, 刘涛. 户外增强现实地理信息系统原型设计与实现[J]. 武汉大学学报(信息科学版), 2007, 32(11):1046-1049. DU Qingyun, LIU Tao. Design and implementation of a prototype outdoor augmented reality GIS[J]. Geomatics and Information Science of Wuhan University, 2007, 32(11):1046-1049.
[2]	孙敏, 陈秀万, 张飞舟, 等. 增强现实地理信息系统[J]. 北京大学学报(自然科学版), 2004, 40(6):906-913. SUN Min, CHEN Xiuwan, ZHANG Feizhou, et al. Augment reality geographical information system[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2004, 40(6):906-913.
[3]	刘经南, 高柯夫. 增强现实及其在导航与位置服务中的应用[J]. 地理空间信息, 2013, 11(2):1-6, 14. LIU Jingnan, GAO Kefu. Augmented reality and its applications for navigation and location based service[J]. Geospatial Information, 2013, 11(2):1-6, 14.
[4]	罗斌, 王涌天, 沈浩, 等. 增强现实混合跟踪技术综述[J]. 自动化学报, 2013, 39(8):1185-1201. LUO Bin, WANG Yongtian, SHEN Hao, et al. Overview of hybrid tracking in augmented reality[J]. Acta Automatica Sinica, 2013, 39(8):1185-1201.
[5]	YOU Xiong, ZHANG Weiwei, MA Meng, et al. Survey on urban warfare augmented reality[J]. ISPRS International Journal of Geo-Information, 2018, 7(2):46.
[6]	刘浩敏, 章国锋, 鲍虎军. 基于单目视觉的同时定位与地图构建方法综述[J]. 计算机辅助设计与图形学学报, 2016, 28(6):855-868. LIU Haomin, ZHANG Guofeng, BAO Hujun. A survey of monocular simultaneous localization and mapping[J]. Journal of Computer-Aided Design & Computer Graphics, 2016, 28(6):855-868.
[7]	曲毅, 李爱光, 徐旺, 等. 基于位姿传感器的户外ARGIS注册技术[J]. 测绘科学技术学报, 2017, 34(1):106-110. QU Yi, LI Aiguang, XU Wang, et al. Outdoor ARGIS registration techniques based on position-posture sensor[J]. Journal of Geomatics Science and Technology, 2017, 34(1):106-110.
[8]	曲毅, 李爱光, 徐旺, 等. 面向ArGIS的多传感器混合跟踪注册研究[J]. 测绘与空间地理信息, 2017, 40(3):114-117, 121. QU Yi, LI Aiguang, XU Wang, et al. Research on multi sensor hybrid tracking registration for ArGIS[J]. Geomatics & Spatial Information Technology, 2017, 40(3):114-117, 121.
[9]	王俊. 户外增强现实GIS的应用研究[D]. 重庆:西南大学, 2014. WANG Jun. Application research on outdoor augmented reality GIS[D]. Chongqing:Southwest University, 2014.
[10]	SUN Min, LIU Lei, HUANG Wei, et al. Interactive registration for augmented reality GIS[C]//Proceedings of 2012 International Conference on Computer Vision in Remote Sensing. Xiamen, China:IEEE, 2012:246-251.
[11]	HUANG Wei, SUN Min, LI Songnian. A 3D GIS-based interactive registration mechanism for outdoor augmented reality system[J]. Expert Systems with Applications, 2016, 55(8):48-58.
[12]	VENTURA J, HÖLLERER T. Wide-area scene mapping for mobile visual tracking[C]//Proceedings of 2012 IEEE International Symposium on Mixed and Augmented Reality. Atlanta, GA:IEEE, 2012:3-12.
[13]	MIDDELBERG S, SATTLER T, UNTZELMANN O, et al. Scalable 6-DOF localization on mobile devices[C]//Proceedings of the 13th European conference on computer vision. Zurich, Switzerland:Springer, 2014:268-283.
[14]	ARTH C, VENTURA J, SCHMALSTIEG D. Geospatial management and utilization of large-scale urban visual reconstructions[C]//Proceedings of the 4th International Conference on Computing for Geospatial Research and Application. San Jose, CA:IEEE, 2013:64-69.
[15]	段利亚. 移动增强现实大范围定位与注册关键技术研究[D]. 武汉:华中科技大学, 2013. DUAN Liya. Study on key technology of wide area localization and registration for mobile augmented reality systems[D]. Wuhan:Huazhong University of Science and Technology, 2013.
[16]	VENTURA J, ARTH C, REITMAYR G, et al. Global localization from monocular SLAM on a mobile phone[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(4):531-539.
[17]	LIU Haomin, ZHANG Guofeng, BAO Hujun. Robust keyframe-based monocular SLAM for augmented reality[C]//Proceedings of 2016 IEEE International Symposium on Mixed and Augmented Reality. Merida, Mexico:IEEE, 2016:1-10.
[18]	LI Peiliang, QIN Tong, HU Botao, et al. Monocular visual-inertial state estimation for mobile augmented reality[C]//Proceedings of 2017 IEEE International Symposium on Mixed and Augmented Reality. Nantes, France:IEEE, 2017:11-21.
[19]	MENOZZI A, CLIPP B, WENGER E, et al. Development of vision-aided navigation for a wearable outdoor augmented reality system[C]//Proceedings of 2014 IEEE/ION Position, Location and Navigation Symposium-PLANS 2014. Monterey, CA:IEEE, 2014:460-472.
[20]	GANS E, ROBERTS D, BENNETT M, et al. Augmented reality technology for day/night situational awareness for the dismounted soldier[C]//Proceedings Volume 9470, Display Technologies and Applications for Defense, Security, and Avionics IX; and Head- and Helmet-Mounted Displays XX. Baltimore, Maryland, United States:SPIE, 2015:947004.
[21]	CHAM T J, CIPTADI A, TAN W C, et al. Estimating camera pose from a single urban ground-view omnidirectional image and a 2D building outline map[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA:IEEE, 2010:366-373.
[22]	CHU Hang, GALLAGHER A, CHEN T. GPS refinement and camera orientation estimation from a single image and a 2D map[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Columbus, OH:IEEE, 2014:171-178.
[23]	GALLAGHER A C. Using vanishing points to correct camera rotation in images[C]//Proceedings of the 2nd Canadian Conference on Computer and Robot Vision. Victoria, BC, Canada:IEEE, 2005:460-467.
[24]	ZHANG Zhengyou. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334.
[25]	LOBO J, DIAS J. Relative pose calibration between visual and inertial sensors[J]. The International Journal of Robotics Research, 2007, 26(6):561-575.
[26]	ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI:IEEE, 2017:2881-2890.
[27]	VON GIOI R G, JAKUBOWICZ J, MOREL J M, et al. LSD:a fast line segment detector with a false detection control[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(4):722-732.
[28]	ZHONG Baojiang, XU Dongsheng, YANG Jiwen. Vertical corner line detection on buildings in quasi-Manhattan world[C]//Proceedings of 2013 IEEE International Conference on Image Processing. Melbourne, VIC, Australia:IEEE, 2013:3064-3068.
[29]	TARDIF J P. Non-iterative approach for fast and accurate vanishing point detection[C]//Proceedings of the 12th International Conference on Computer Vision. Kyoto, Japan:IEEE, 2009:1250-1257.
[30]	LEPETIT V, MORENO-NOGUER F, FUA P. EPnP:an accurate O(n) solution to the PnP problem[J]. International Journal of Computer Vision, 2009, 81(2):155-166.
[31]	CHULLIAT A, MACMILLAN S, ALKEN P, et al. The US/UK world magnetic model for 2015-2020:technical report[R]. Boulder, CO:NOAA, 2015.
[32]	ARTH C, PIRCHHEIM C, VENTURA J, et al. Instant outdoor localization and SLAM initialization from 2.5D maps[J]. IEEE Transactions on Visualization and Computer Graphics, 2015, 21(11):1309-1318.
[33]	ARTH C, PIRCHHEIM C, VENTURA J, et al. Global 6DOF pose estimation from untextured 2D city models[J]. Computer Science,2015,25(1):1-8.