A semantic enhancement method for photorealistic mesh model based on local parameterization

doi:10.11947/j.AGCS.2020.20180588

Abstract

Abstract: With the advances in structure-from-motion and multi-view stereo, state-of-the-art oblique photogrammetric solutions can obtain city-scale photorealistic mesh models automatically. However, the mesh models are lack of fine geometric structure and semantic free. Aiming at solving this issue, it is proposed that a semantic enhancement method for photorealistic mesh models based on local surface parametrization. The basic idea behind the proposed method is that, through the representation of surface tree, it is converted that the seamless fusion of semantic components and photogrammetric mesh models to a replacing operation in a local area. The two 3D models are parametrized to 2D space in the local region and seamless merged and replaced in the UV space by 2D constrained delaunay triangulation (CDT). The replaced semantic components are then reversed transform to 3D space, which finalize the semantic enhancement automatically. Experiments on oblique images in Shenzhen reveal that the proposed method can effectively realize the automatic seamless fusion of semantic components with an open boundary and photorealistic mesh models. Compared with the commercial software Maya, based on the method of insertion and fusion, the proposed method has practical value for improving modeling efficiency.

Key words: 3D surface structure tree, local parameterization, triangulation model, semantic enhancement, oblique photogrammetry

CLC Number:

P237

WANG Libin, HU Han, ZHU Qing, DING Yulin, CHEN Min. A semantic enhancement method for photorealistic mesh model based on local parameterization[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(2): 225-234.

References

[1] NEX F, GERKE M, REMONDINO F, et al. ISPRS benchmark for multi-platform photogrammetry[C]//Proceedings of ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Munich, Germany:[s.n.], 2015:135-142.
[2] BARROW C. Oblique imagery:the standard for mapping[EB/OL]. (2014-04-23). https://www.gim-international.com/content/article/oblique-imagery-the-standard-for-mapping.
[3] BILJECKI F, STOTER J, LEDOUX H, et al. Applications of 3D city models:state of the art review[J]. ISPRS International Journal of Geo-Information, 2015, 4(4):2842-2889.
[4] KOLBE T H, GRÖGER G, PLVMER L. CityGML:Interoperable access to 3D city models[M]//VAN OOSTEROM P, ZLATANOVA S, FENDEL E M. Geo-information for Disaster Management. Berlin, Heidelberg:Springer, 2005:883-899.
[5] FUHRMANN S, GOESELE M. Fusion of depth maps with multiple scales[J]. ACM Transactions on Graphics, 2011, 30(6):148.
[6] 肖雄武, 郭丙轩, 李德仁, 等. 一种具有仿射不变性的倾斜影像快速匹配方法[J]. 测绘学报, 2015, 44(4):414-421. DOI:10.11947/j.AGCS.2015.20140048. XIAO Xiongwu, GUO Bingxuan, LI Deren, et al. A quick and affine invariance matching method for oblique images[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(4):414-421. DOI:10.11947/j.AGCS.2015.20140048.
[7] YANG Qingxiong. Stereo matching using tree filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(4):834-846.
[8] 张春森, 张卫龙, 郭丙轩, 等. 倾斜影像的三维纹理快速重建[J]. 测绘学报, 2015, 44(7):782-790. DOI:10.11947/j.AGCS.2015.20140341. ZHANG Chunsen, ZHANG Weilong, GUO Bingxuan, et al. Rapidly 3D texture reconstruction based on oblique photography[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(7):782-790. DOI:10.11947/j.AGCS.2015.20140341.
[9] 李劲澎. 基于全局式运动恢复结构的无人机影像位姿估计关键技术研究[D]. 郑州:信息工程大学, 2017. LI Jinpeng. Key technology of position and orientation estimation for UAV images based on global structure from motion[D]. Zhengzhou:Information Engineering University, 2017.
[10] HEINLY J, SCHÖNBERGER J L, DUNN E, et al. Reconstructing the world^* in six days[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:3287-3295.
[11] HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2):328-341.
[12] FURUKAWA Y, PONCE J. Accurate, dense, and robust multiview stereopsis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(8):1362-1376.
[13] SCHÖNBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:4104-4113.
[14] BERGER M, TAGLIASACCHI A, SEVERSKY L, et al. State of the art in surface reconstruction from point clouds[C]//Proceedings of Eurographics 2014-State of the Art Reports. Strasbourg, France:[s.n.], 2014:161-185.
[15] HAALA N, KADA M. An update on automatic 3D building reconstruction[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2010, 65(6):570-580.
[16] MUSIALSKI P, WONKA P, ALIAGA D G, et al. A survey of urban reconstruction[J]. Computer Graphics Forum, 2013, 32(6):146-177.
[17] VERDIE Y, LAFARGE F, ALLIEZ P. LOD generation for urban scenes[J]. ACM Transactions on Graphics, 2015, 34(3):30.
[18] MONSZPART A, MELLADO N, BROSTOW G J, et al. RAPter:rebuilding man-made scenes with regular arrangements of planes[J]. ACM Transactions on Graphics, 2015, 34(4):103.
[19] CHAUVE A L, LABATUT P, PONS J P. Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA:IEEE, 2010:1261-1268.
[20] VU H H, LABATUT P, PONS J P, et al. High accuracy and visibility-consistent dense multiview stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(5):889-901.
[21] BERGER M, TAGLIASACCHI A, SEVERSKY L, et al. State of the art in surface reconstruction from point clouds[C]//Proceedings of Eurographics 2014-State of the Art Reports. Strasbourg, France:[s.n.], 2014:161-185.
[22] LAFARGE F, ALLIEZ P. Surface reconstruction through point set structuring[J]. Computer Graphics Forum, 2013, 32(2):225-234.
[23] ORTHUBER E, AVBELJ J. 3D building reconstruction from LiDAR point clouds by adaptive dual contouring[C]//Proceedings of ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Munich, Germany:[s.n.], 2015:157-164.
[24] SCHMIDT R, SINGH K. Sketch-based procedural surface modeling and compositing using surface trees[J]. Computer Graphics Forum, 2008, 27(2):321-330.
[25] LÉVY B, PETITJEAN S, RAY N, et al. Least squares conformal maps for automatic texture atlas generation[J]. ACM Transactions on Graphics, 2002, 21(3):362-371.
[26] JACOBSON A, BARAN I, KAVAN L, et al. Fast automatic skinning transformations[J]. ACM Transactions on Graphics, 2012, 31(4):77.
[27] WYVILL B, GUY A, GALIN E. Extending the CSG tree. warping, blending and boolean operations in an implicit surface modeling system[J]. Computer Graphics Forum, 1999, 18(2):149-158.
[28] DO CARMO M P. Differential geometry of curves and surfaces:revised and updated[M]. 2nd ed. Mineola, New York:Dover Publications, 2016.
[29] CRANE K, WEISCHEDEL C, WARDETZKY M. The heat method for distance computation[J]. Communications of the ACM, 2017, 60(11):90-99.
[30] SCHMIDT R, GRIMM C, WYVILL B. Interactive decal compositing with discrete exponential maps[J]. ACM Transactions on Graphics, 2006, 25(3):605-613.
[31] 胡翰. Oblique Sketch:倾斜摄影测量交互式建模软件[EB/OL]. (2016-10-29). http://www.vrlab.org.cn/. HU Han. Oblique Sketch:an interactive modeling software of oblique photogrammetry[EB/OL]. (2016-10-29). http://www.vrlab.org.cn/.