Motivation 

Despite the rapid progress of deep learning (DL) in 3D computer vision, 3D shape completion in point clouds remains fundamentally unsolved challenge. This difficulty stems from the inherent irregularities of point cloud data, including a) their unordered and unstructured nature; b) the variability of sensor modalities and scene contexts (e.g., indoor vs. outdoor, LiDAR vs. RGB-D), which causes severe domain shifts; and c) the scarcity and imbalance of annotated 3D data compared to large-scale
2D image datasets, resulting in the persistent long-tail problem.

Before the DL era, geometric-feature-based approaches demonstrated remarkable success in modeling local shape priors and inferring missing geometry through hand-crafted descriptors. However, these methods lacked robustness and failed to generalize beyond the datasets they were designed for. While DL architectures promised to overcome such limitations through learned representations [2,3], they still struggle to encode fine-grained geometric regularities necessary for reliable shape completion. This shortcoming often leads researchers to geometric shortcuts, such as projecting 3D point cloud into 2.5D [1] depth maps or relying on symmetry assumptions [3].

Our preliminary research indicates that integrating explicit geometric features (e.g., anisotropy, roughness, planarity) with learned features substantially improves the network’s performance for semantic segmentation accelerating convergence and enhancing robustness across datasets (see here: [1] and another paper under-review). We are convinced that this idea can be transferred to even more geometric-oriented task of shape completion. Our goal is to leverage geometric priors as structural regularizer for shape completion, aiming to bridge the gap between accuracy and generalization in complex environments from aerial to terrestrial acquisition.

Objectives 

Specific tasks of the team:

• Develop a method, GF4Shape, that fuses geometric features and learned features based on the diffusion approach. (M1 - Milestone 1)
• Design geometric-feature-based regularizer for shape completion and compare to the M1 results. (M2)
• Evaluate the GF4Shape shape completion method on indoor and urban outdoor environments (see Data). (M3)

Data 

Experiments shall be conducted on our in-house dataset (soon to be published) and on the benchmarks of computer vision community, such as KITTI / KITTI-360, SemanticKITTI, nuScene, RoofDiffusion, Building-PCC, TUM2TWIN and PCN.

Requirements

• Computer vision, machine learning, data science, photogrammetry or similar background is required.
• Self-motivated and strongly interested in publishing top venues such as CVPR, ECCV, NeurIPS, and similar (Previous publications is a plus).
• Familiar with Pytorch or Tensorflow deep learning framework.

Apply to this project here

References 

[1] Lo, K.S.H., Peters, J. and Spellman, E., 2024, RoofDiffusion: Constructing roofs from severely
corrupted point data via diffusion. ECCV24.
[2] Chen, Z., Wang, Y., Nan, L. and Zhu, X.X., 2025. Parametric Point Cloud Completion for Polygonal Surface Reconstruction. CVPR25.
[3]Yuan, W., Khot, T., Held, D., Mertz, C. and Hebert, M., 2018, PCN: Point completion network. 3DV18
[4] Xia, Y., Xia, Y., Li, W., Song, R., Cao, K. and Stilla, U., 2021, Asfm-net: Asymmetrical siamese feature matching network for point completion. ACM21.
[5] Tan, Y., Wysocki, O., Hoegner, L. and Stilla, U., 2023, Classifying point clouds at the facade-level using geometric features and deep learning networks. 3DGeoInfo23.