Alert button
Picture for Wenchao Ma

Wenchao Ma

Alert button

HoW-3D: Holistic 3D Wireframe Perception from a Single Image

Aug 19, 2022
Wenchao Ma, Bin Tan, Nan Xue, Tianfu Wu, Xianwei Zheng, Gui-Song Xia

Figure 1 for HoW-3D: Holistic 3D Wireframe Perception from a Single Image
Figure 2 for HoW-3D: Holistic 3D Wireframe Perception from a Single Image
Figure 3 for HoW-3D: Holistic 3D Wireframe Perception from a Single Image
Figure 4 for HoW-3D: Holistic 3D Wireframe Perception from a Single Image

This paper studies the problem of holistic 3D wireframe perception (HoW-3D), a new task of perceiving both the visible 3D wireframes and the invisible ones from single-view 2D images. As the non-front surfaces of an object cannot be directly observed in a single view, estimating the non-line-of-sight (NLOS) geometries in HoW-3D is a fundamentally challenging problem and remains open in computer vision. We study the problem of HoW-3D by proposing an ABC-HoW benchmark, which is created on top of CAD models sourced from the ABC-dataset with 12k single-view images and the corresponding holistic 3D wireframe models. With our large-scale ABC-HoW benchmark available, we present a novel Deep Spatial Gestalt (DSG) model to learn the visible junctions and line segments as the basis and then infer the NLOS 3D structures from the visible cues by following the Gestalt principles of human vision systems. In our experiments, we demonstrate that our DSG model performs very well in inferring the holistic 3D wireframes from single-view images. Compared with the strong baseline methods, our DSG model outperforms the previous wireframe detectors in detecting the invisible line geometry in single-view images and is even very competitive with prior arts that take high-fidelity PointCloud as inputs on reconstructing 3D wireframes.

* To appear in IEEE 3DV 2022. Code and Dataset are available at https://github.com/Wenchao-M/HoW-3D 
Viaarxiv icon

Inverse-Designed Meta-Optics with Spectral-Spatial Engineered Response to Mimic Color Perception

Apr 28, 2022
Chris Munley, Wenchao Ma, Johannes E. Fröch, Quentin A. A. Tanguy, Elyas Bayati, Karl F. Böhringer, Zin Lin, Raphaël Pestourie, Steven G. Johnson, Arka Majumdar

Figure 1 for Inverse-Designed Meta-Optics with Spectral-Spatial Engineered Response to Mimic Color Perception
Figure 2 for Inverse-Designed Meta-Optics with Spectral-Spatial Engineered Response to Mimic Color Perception
Figure 3 for Inverse-Designed Meta-Optics with Spectral-Spatial Engineered Response to Mimic Color Perception
Figure 4 for Inverse-Designed Meta-Optics with Spectral-Spatial Engineered Response to Mimic Color Perception

Meta-optics have rapidly become a major research field within the optics and photonics community, strongly driven by the seemingly limitless opportunities made possible by controlling optical wavefronts through interaction with arrays of sub-wavelength scatterers. As more and more modalities are explored, the design strategies to achieve desired functionalities become increasingly demanding, necessitating more advanced design techniques. Herein, the inverse-design approach is utilized to create a set of single-layer meta-optics that simultaneously focus light and shape the spectra of focused light without using any filters. Thus, both spatial and spectral properties of the meta-optics are optimized, resulting in spectra that mimic the color matching functions of the CIE 1931 XYZ color space, which links the distributions of wavelengths in light and the color perception of a human eye. Experimental demonstrations of these meta-optics show qualitative agreement with the theoretical predictions and help elucidate the focusing mechanism of these devices.

Viaarxiv icon