Flex-Convolution

Million-Scale Point-Cloud Learning Beyond Grid-Worlds

Fabian Groh, Patrick Wieschollek and Hendrik P. A. Lensch
University of Tübingen
Asian Conference on Computer Vision (ACCV) 2018

Actually, you are supposed to see a video here. To display this content (source: www.xyz.de), please click the "Accept" button below. Please note that by playing the video, in addition to cookies required for video playback, third-party cookies may be set for targeting and advertising purposes, and data may be linked with other services of the third-party provider. Further information and options to withdraw your consent can be found in our privacy policy.

Abstract

Traditional convolution layers are specifically designed to exploit the natural data representation of images -- a fixed and regular grid. However, unstructured data like 3D point clouds containing irregular neighborhoods constantly breaks the grid-based data assumption. Therefore applying best-practices and design choices from 2D-image learning methods towards processing point clouds are not readily possible. In this work, we introduce a natural generalization flex-convolution of the conventional convolution layer along with an efficient GPU implementation. We demonstrate competitive performance on rather small benchmark sets using fewer parameters and lower memory consumption and obtain significant improvements on a million-scale real-world dataset. Ours is the first which allows to efficiently process 7 million points concurrently.

Content

Our network can handle up to handle inputs up to 7 on a NVIDIA 1080Ti in a single forward pass with the same accuracy.

It scales beyond, up to 17 Million points on a V100 GPU.

It therefore processes entire rooms at full-resolution without the need of sub-sampling.

Semantic point cloud segmentation produced as raw outputs of our proposed network from the held-out validation set. In this point-based rendering, surfaces might not be illustrated as opaque. Interestingly, in room C our network classifies the beanbag as “sofa”, while its ground-truth annotation is “chair”. For more results please refer to the accompanying video.

More Resources

Paper

Code

Bibtex

@inproceedings{accv2018/Groh,
author = {Fabian Groh and Patrick Wieschollek and Hendrik P. A. Lensch },
  title = {Flex-Convolution (Million-Scale Pointcloud Learning Beyond Grid-Worlds)},
  booktitle = {Asian Conference on Computer Vision (ACCV)},
  month = {Dezember},
  year = {2018}
}

Million-Scale Point-Cloud Learning Beyond Grid-Worlds

External content