A fast fusion object determination method for multi-path video and three-dimensional GIS scene

doi:10.11947/j.AGCS.2020.20190282

Abstract

Abstract: Virtual and real fusion technology is one of the key links in video enhancement of 3D geographical scene, which plays an important role in improving the real immersive visual experience. The existing video projection fusion method usually needs to traverse all the rendered objects in the new scene when the 3D scene is updated dynamically, in order to determine the objects that need to be fused in the camera range, which resulting in a long fusion process and low efficiency. A fast fusion method of multi-channel video and 3D GIS scene considering topological relationship is proposed. Firstly, a dynamic topology network model was established for video objects and rendering objects to accelerate the judgment of spatial relationship. Secondly, the topology relationship between video and rendering objects within 3D scene was established based on the topology network model, and the recorded topological relationship is used to determine and fuse the object quickly. Finally, the three-dimensional scene of Tengzhou City is verified. The experimental results show that the proposed method has the advantages of less time-consuming, high efficiency, greatly reducing the computational cost of fusion, and has good feasibility and robustness in the fusion of video and 3D GIS scene.

Key words: video, topology information, fusion object, 3D GIS scene, dynamic update

CLC Number:

P208

LIU Zhendong, DAI Zhaoxin, LI Chengming, LIU Xiaoli. A fast fusion object determination method for multi-path video and three-dimensional GIS scene[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(5): 632-643.

References

[1] MILOSAVLJEVIĆ A, DIMITRIJEVIĆ A, RANČIĆ D. GIS-augmented video surveillance[J]. International Journal of Geographical Information Science, 2010, 24(9): 1415-1433.
[2] MILOSAVLJEVIĆ A, RANČIĆ D, DIMITRIJEVIĆ A, et al. Integration of GIS and video surveillance[J]. International Journal of Geographical Information Science, 2016, 30(10): 2089-2107.
[3] PAN Chengwei, CHEN Yisong, WANG Guoping. Virtual-real fusion with dynamic scene from videos[C]//Proceedings of 2016 International Conference on Cyberworlds (CW). Chongqing: IEEE, 2016: 312-325.
[4] JIAN Hongdeng, LIAO Jingjuan, FAN Xiangtao, et al. Augmented virtual environment: fusion of real-time video and 3D models in the digital earth system[J]. International Journal of Digital Earth, 2017, 10(12) 1177-1196.
[5] DECAMP P, SHAW G, KUBAT R, et al. An immersive system for browsing and visualizing surveillance video[C]//Proceedings of the 18th ACM International Conference on Multimedia. Firenze: ACM, 2010: 371-380.
[6] 周凡. 视频影像增强虚拟三维场景的注册与渲染方法研究[J]. 测绘学报, 2019, 48(6): 801. DOI: 10.11947/j.AGCS.2019.20180315. ZHOU Fan. Research on registration and rendering method of video to enhance 3D scene[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 801. DOI: 10.11947/j.AGCS.2019.20180315.
[7] SAWHNEY HS, ARPA A, KUMAR R, et al. Video flashlights: real time rendering of multiple videos for immersive model visualization[C]//Proceedings of the 13th Eurographics Workshop on Rendering. Pisa, Italy: ACM, 2002: 157-168.
[8] NEUMANN U, YOU S Y, HU J H, et al. Augmented virtual environments (AVE): dynamic fusion of imagery and 3D models[C]//Proceedings of 2003 IEEE Virtual Reality. Los Angeles, CA: IEEE, 2003: 61-67.
[9] DE HAAN G, PIGUILLET H, POST F H. Spatial navigation for context-aware video surveillance[J]. IEEE Computer Graphics and Applications, 2010, 30(5): 20-31.
[10] HU Jinhui. Integrating complementary information for photorealistic representation[D]. Los Angeles: University of Southern California, 2009.
[11] 周颐, 孟明, 吴威, 等. 基于视频模型的虚拟现实视频融合系统[J]. 系统仿真学报, 2018, 30(7): 2250-2557. ZHOU Yi, MENG Ming, WU Wei, et al. Virtual-reality video fusion system based on video model[J]. Journal of System Simulation, 2018, 30(7): 2550-2557.
[12] WANG Y, KRUM D M, COELHO E M, et al. Contextualized videos: combining videos with environment models to support situational understanding[J]. IEEE Transactions on Visualization and Computer Graphics, 2007, 13(6): 1568-1575.
[13] 马原野, 赵刚, 何彬. 3D GIS与多视频融合系统的设计与实现[J]. 计算机应用与软件, 2012, 29(6): 109-112, 117. MA Yuanye, ZHAO Gang, HE Bin. Design and implementation of a fused system with 3DGIS and multiple-videos[J]. Computer Applications and Software, 2012, 29(6): 109-112, 117.
[14] DE HAAN G, SCHEUER J, DE VRIES R, et al. Egocentric navigation for video surveillance in 3D virtual environments[C]//Proceedings of 2009 IEEE Symposium on 3D User Interfaces. Lafayette, LA: IEEE, 2009.
[15] CORRAL-SOTO E R, TAL R, WANG Langyue, et al. 3D town: the automatic urban awareness project[C]//Proceedings of 2012 Ninth Conference on Computer and Robot Vision (CRV). Toronto: IEEE, 2012: 433-440.
[16] MOWER J E. Creating and delivering augmented scenes[J]. International Journal of Geographical Information Science, 2009, 23(8): 993-1011.
[17] HSU S, SAMARASEKERA S, KUMAR R, et al. Pose estimation, model refinement, and enhanced visualization using video[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Hilton Head Island: IEEE, 2000: 488-495.
[18] SEGAL M, KOROBKIN C, VAN WIDENFEL T R, et al. Fast shadows and lighting effects using texture mapping[J]. ACM SIGGRAPH Computer Graphics, 1992, 26(2): 249-252.
[19] 简洪登, 范湘涛. 基于GLSL的多重视频纹理映射与融合[J]. 计算机工程与设计, 2014, 35(11): 3873-3878. JIAN Hongdeng, FAN Xiangtao. Multiple video texture mapping and fusion based on OpenGL shading language[J]. Computer Engineering and Design, 2014, 35(11): 3873-3878.
[20] GHADIRIAN P, BISHOP I D. Integration of augmented reality and GIS: a new approach to realistic landscape visualisation[J]. Landscape and Urban Planning, 2008, 86(3-4): 226-232.
[21] 王行风, 汪云甲. 一种顾及拓扑关系的室内三维模型组织和调度方法[J]. 武汉大学学报(信息科学版), 2017, 42(1): 35-42. WANG Xingfeng, WANG Yunjia. Organization and scheduling of indoor three-dimensional geometric model based on spatial topological relation[J]. Geomatics and Information Science of Wuhan University, 2017, 42(1): 35-42.
[22] 龚俊, 朱庆, 章汉武, 等. 基于R树索引的三维场景细节层次自适应控制方法[J]. 测绘学报, 2011, 40(4): 531-534. GONG Jun, ZHU Qing, ZHANG Hanwu, et al. An adaptive method for 3D city model based on extended 3D R-tree[J]. Acta Geodaetica et Cartographica Sinica, 2011, 40(4): 531-534.
[23] 朱庆, 陈兴旺, 丁雨淋, 等. 视觉感知驱动的三维城市场景数据组织与调度方法[J]. 西南交通大学学报, 2017, 52(5): 869-876. ZHU Qing, CHEN Xingwang, DING Yulin, et al. Organization and scheduling method of 3D urban scene data driven by visual perception[J]. Journal of Southwest Jiaotong University, 2017, 52(5): 869-876.
[24] CHEN C F, BOLAS M, SUMA E. Real-time 3D rendering using depth-based geometry reconstruction and view-dependent texture mapping[C]//Proceedings of ACM SIGGRAPH 2016. Anaheim, CA:ACM, 2016: 213-227.
[25] CLARKE B L. A calculus of individuals based on “connection”[J]. Notre Dame Journal of Formal Logic, 1981, 22(3): 204-218.
[26] LEWIS P, FOTHERINGHAM S, WINSTANLEY A. Spatial video and GIS[J]. International Journal of Geographical Information Science, 2011, 25(5): 697-716.