PWR-Align: Leveraging Part-Whole Relationships for Part-wise Rigid Point Cloud Registration in Mixed Reality Applications

We present an efficient and robust point cloud registration (PCR) workflow for part-wise rigid point cloud alignment using the Microsoft HoloLens 2. Point Cloud Registration (PCR) is an important problem in Augmented and Mixed Reality use cases, and we present a study for a special class of non-rigid transformations. Many commonly encountered objects are composed of rigid parts that move relative to one another about joints resulting in non-rigid deformation of the whole object such as robots with manipulators, and machines with hinges. The workflow presented allows us to register the point cloud with various configurations of the point cloud.

Point Cloud Registration of non-rigid objects in sparse 3D Scans with applications in Mixed Reality

Point Cloud Registration is the problem of aligning the corresponding points of two 3D point clouds referring to the same object. The challenges include dealing with noise and partial match of real-world 3D scans. For non-rigid objects, there is an additional challenge of accounting for deformations in the object shape that happen to the object in between the two 3D scans. In this project, we study the problem of non-rigid point cloud registration for use cases in the Augmented/Mixed Reality domain. We focus our attention on a special class of non-rigid deformations that happen in rigid objects with parts that move relative to one another about joints, for example, robots with hands and machines with hinges. We propose an efficient and robust point-cloud registration workflow for such objects and evaluate it on real-world data collected using Microsoft Hololens 2, a leading Mixed Reality Platform.