Abstract: An automatic urban building extraction method for high-resolution remote-sensing imagery, which combines building segmentation based on neighbor total variations with object-oriented analysis, is presented in this paper. Aimed at different extraction complexity from various buildings in the segmented image, a hierarchical building extraction strategy with multi-feature fusion is adopted. Firstly, we extract some rectangle buildings which remain intact after segmentation through shape analysis. Secondly, in order to ensure each candidate building target to be independent, multidirectional morphological road-filtering algorithm is designed which can separate buildings from the neighboring roads with similar spectrum. Finally, we take the extracted buildings and the excluded non-buildings as samples to establish probability model respectively, and Bayesian discriminating classifier is used for making judgment of the other candidate building objects to get the ultimate extraction result. The experimental results show that the approach is able to detect buildings with different structure and spectral features in the same image. The results of performance evaluation also support the robustness and precision of the approach developed.