PointInverter: Point Cloud Reconstruction and Editing via a Generative Model with Shape Priors

Jaeyeon Kim1          Binh-Son Hua3          Duc Thanh Nguyen4          Sai-Kit Yeung1

1Hong Kong University of Science and Technology 2VinAI Research
3VinUniversity 4Deakin University

IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023

Abstract

In this paper, we propose a new method for mapping a 3D point cloud to the latent space of a 3D generative adversarial network. Our generative model for 3D point clouds is based on SP-GAN, a state-of-the-art sphere-guided 3D point cloud generator. We derive an efficient way to encode an input 3D point cloud to the latent space of the SP-GAN. Our point cloud encoder can resolve the point ordering issue during inversion, and thus can determine the correspondences be- tween points in the generated 3D point cloud and those in the canonical sphere used by the generator. We show that our method outperforms previous GAN inversion methods for 3D point clouds, achieving state-of-the-art results both quanti- tatively and qualitatively.


Materials
Paper
Poster

Our network

Overview of PointInverter network

Results

Inversion Reconstruction. The inverted point clouds by pointinverter are very similar to the target input data.



Dense correspondence. The reconstuction sample using our inversion model maintains the dense correspondence with SP-GAN.

Citation
@article{jy-pointinverter-wacv23,
            title = {PointInverter: Point Cloud Reconstruction and Editing via a Generative Model with Shape Priors},
            author = {Jaeyeon Kim and Binh-Son Hua and Duc Thanh Nguyen and Sai-Kit Yeung},
            booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
            year = {2023}
        }

This research project is partially supported by an internal grant from HKUST (R9429).