Alert button
Picture for Qian Zheng

Qian Zheng

Alert button

Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN

May 31, 2023
Yangfan Hu, Qian Zheng, Xudong Jiang, Gang Pan

Figure 1 for Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN
Figure 2 for Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN
Figure 3 for Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN
Figure 4 for Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN

Spiking neural networks (SNNs) have shown advantages in computation and energy efficiency over traditional artificial neural networks (ANNs) thanks to their event-driven representations. SNNs also replace weight multiplications in ANNs with additions, which are more energy-efficient and less computationally intensive. However, it remains a challenge to train deep SNNs due to the discrete spike function. A popular approach to circumvent this challenge is ANN-to-SNN conversion. However, due to the quantization error and accumulating error, it often requires lots of time steps (high inference latency) to achieve high performance, which negates SNN's advantages. To this end, this paper proposes Fast-SNN that achieves high performance with low latency. We demonstrate the equivalent mapping between temporal quantization in SNNs and spatial quantization in ANNs, based on which the minimization of the quantization error is transferred to quantized ANN training. With the minimization of the quantization error, we show that the sequential error is the primary cause of the accumulating error, which is addressed by introducing a signed IF neuron model and a layer-wise fine-tuning mechanism. Our method achieves state-of-the-art performance and low latency on various computer vision tasks, including image classification, object detection, and semantic segmentation. Codes are available at: https://github.com/yangfan-hu/Fast-SNN.

* Accepted to IEEE Transactions on Pattern Analysis and Machine Intelligence 
Viaarxiv icon

DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering

Mar 28, 2023
Zongrui Li, Qian Zheng, Boxin Shi, Gang Pan, Xudong Jiang

Figure 1 for DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering
Figure 2 for DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering
Figure 3 for DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering
Figure 4 for DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering

Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguity brought by the unknown light. Although the ambiguity is alleviated on non-Lambertian objects, the problem is still difficult to solve for more general objects with complex shapes introducing irregular shadows and general materials with complex reflectance like anisotropic reflectance. To exploit cues from shadow and reflectance to solve UPS and improve performance on general materials, we propose DANI-Net, an inverse rendering framework with differentiable shadow handling and anisotropic reflectance modeling. Unlike most previous methods that use non-differentiable shadow maps and assume isotropic material, our network benefits from cues of shadow and anisotropic reflectance through two differentiable paths. Experiments on multiple real-world datasets demonstrate our superior and robust performance.

* Accepted by CVPR 2023 
Viaarxiv icon

Evaluating the Efficacy of Skincare Product: A Realistic Short-Term Facial Pore Simulation

Feb 23, 2023
Ling Li, Bandara Dissanayake, Tatsuya Omotezako, Yunjie Zhong, Qing Zhang, Rizhao Cai, Qian Zheng, Dennis Sng, Weisi Lin, Yufei Wang, Alex C Kot

Figure 1 for Evaluating the Efficacy of Skincare Product: A Realistic Short-Term Facial Pore Simulation
Figure 2 for Evaluating the Efficacy of Skincare Product: A Realistic Short-Term Facial Pore Simulation
Figure 3 for Evaluating the Efficacy of Skincare Product: A Realistic Short-Term Facial Pore Simulation
Figure 4 for Evaluating the Efficacy of Skincare Product: A Realistic Short-Term Facial Pore Simulation

Simulating the effects of skincare products on face is a potential new way to communicate the efficacy of skincare products in skin diagnostics and product recommendations. Furthermore, such simulations enable one to anticipate his/her skin conditions and better manage skin health. However, there is a lack of effective simulations today. In this paper, we propose the first simulation model to reveal facial pore changes after using skincare products. Our simulation pipeline consists of 2 steps: training data establishment and facial pore simulation. To establish training data, we collect face images with various pore quality indexes from short-term (8-weeks) clinical studies. People often experience significant skin fluctuations (due to natural rhythms, external stressors, etc.,), which introduces large perturbations in clinical data. To address this problem, we propose a sliding window mechanism to clean data and select representative index(es) to represent facial pore changes. Facial pore simulation stage consists of 3 modules: UNet-based segmentation module to localize facial pores; regression module to predict time-dependent warping hyperparameters; and deformation module, taking warping hyperparameters and pore segmentation labels as inputs, to precisely deform pores accordingly. The proposed simulation is able to render realistic facial pore changes. And this work will pave the way for future research in facial skin simulation and skincare product developments.

* 6 pages, 7 figures 
Viaarxiv icon

NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo

Aug 19, 2022
Zongrui Li, Qian Zheng, Feishi Wang, Boxin Shi, Gang Pan, Xudong Jiang

Figure 1 for NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo
Figure 2 for NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo
Figure 3 for NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo
Figure 4 for NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo

Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguity brought by unknown light. Existing solutions alleviate the ambiguity by either explicitly associating reflectance to light conditions or resolving light conditions in a supervised manner. This paper establishes an implicit relation between light clues and light estimation and solves UPS in an unsupervised manner. The key idea is to represent the reflectance as four neural intrinsics fields, i.e., position, light, specular, and shadow, based on which the neural light field is implicitly associated with light clues of specular reflectance and cast shadow. The unsupervised, joint optimization of neural intrinsics fields can be free from training data bias as well as accumulating error, and fully exploits all observed pixel values for UPS. Our method achieves a superior performance advantage over state-of-the-art UPS methods on public and self-collected datasets, under regular and challenging setups. The code will be released soon.

Viaarxiv icon

TinyLight: Adaptive Traffic Signal Control on Devices with Extremely Limited Resources

May 01, 2022
Dong Xing, Qian Zheng, Qianhui Liu, Gang Pan

Figure 1 for TinyLight: Adaptive Traffic Signal Control on Devices with Extremely Limited Resources
Figure 2 for TinyLight: Adaptive Traffic Signal Control on Devices with Extremely Limited Resources
Figure 3 for TinyLight: Adaptive Traffic Signal Control on Devices with Extremely Limited Resources
Figure 4 for TinyLight: Adaptive Traffic Signal Control on Devices with Extremely Limited Resources

Recent advances in deep reinforcement learning (DRL) have largely promoted the performance of adaptive traffic signal control (ATSC). Nevertheless, regarding the implementation, most works are cumbersome in terms of storage and computation. This hinders their deployment on scenarios where resources are limited. In this work, we propose TinyLight, the first DRL-based ATSC model that is designed for devices with extremely limited resources. TinyLight first constructs a super-graph to associate a rich set of candidate features with a group of light-weighted network blocks. Then, to diminish the model's resource consumption, we ablate edges in the super-graph automatically with a novel entropy-minimized objective function. This enables TinyLight to work on a standalone microcontroller with merely 2KB RAM and 32KB ROM. We evaluate TinyLight on multiple road networks with real-world traffic demands. Experiments show that even with extremely limited resources, TinyLight still achieves competitive performance. The source code and appendix of this work can be found at \url{https://bit.ly/38hH8t8}.

* Accepted by IJCAI 2022 (Long Oral) 
Viaarxiv icon

Automatic Facial Skin Feature Detection for Everyone

Mar 30, 2022
Qian Zheng, Ankur Purwar, Heng Zhao, Guang Liang Lim, Ling Li, Debasish Behera, Qian Wang, Min Tan, Rizhao Cai, Jennifer Werner, Dennis Sng, Maurice van Steensel, Weisi Lin, Alex C Kot

Figure 1 for Automatic Facial Skin Feature Detection for Everyone
Figure 2 for Automatic Facial Skin Feature Detection for Everyone
Figure 3 for Automatic Facial Skin Feature Detection for Everyone
Figure 4 for Automatic Facial Skin Feature Detection for Everyone

Automatic assessment and understanding of facial skin condition have several applications, including the early detection of underlying health problems, lifestyle and dietary treatment, skin-care product recommendation, etc. Selfies in the wild serve as an excellent data resource to democratize skin quality assessment, but suffer from several data collection challenges.The key to guaranteeing an accurate assessment is accurate detection of different skin features. We present an automatic facial skin feature detection method that works across a variety of skin tones and age groups for selfies in the wild. To be specific, we annotate the locations of acne, pigmentation, and wrinkle for selfie images with different skin tone colors, severity levels, and lighting conditions. The annotation is conducted in a two-phase scheme with the help of a dermatologist to train volunteers for annotation. We employ Unet++ as the network architecture for feature detection. This work shows that the two-phase annotation scheme can robustly detect the accurate locations of acne, pigmentation, and wrinkle for selfie images with different ethnicities, skin tone colors, severity levels, age groups, and lighting conditions.

* Accepted by the conference of Electronic Imaging (EI) 2022 
Viaarxiv icon

On Convergence of Gradient Expected Sarsa($λ$)

Dec 14, 2020
Long Yang, Gang Zheng, Yu Zhang, Qian Zheng, Pengfei Li, Gang Pan

Figure 1 for On Convergence of Gradient Expected Sarsa($λ$)
Figure 2 for On Convergence of Gradient Expected Sarsa($λ$)
Figure 3 for On Convergence of Gradient Expected Sarsa($λ$)
Figure 4 for On Convergence of Gradient Expected Sarsa($λ$)

We study the convergence of $\mathtt{Expected~Sarsa}(\lambda)$ with linear function approximation. We show that applying the off-line estimate (multi-step bootstrapping) to $\mathtt{Expected~Sarsa}(\lambda)$ is unstable for off-policy learning. Furthermore, based on convex-concave saddle-point framework, we propose a convergent $\mathtt{Gradient~Expected~Sarsa}(\lambda)$ ($\mathtt{GES}(\lambda)$) algorithm. The theoretical analysis shows that our $\mathtt{GES}(\lambda)$ converges to the optimal solution at a linear convergence rate, which is comparable to extensive existing state-of-the-art gradient temporal difference learning algorithms. Furthermore, we develop a Lyapunov function technique to investigate how the step-size influences finite-time performance of $\mathtt{GES}(\lambda)$, such technique of Lyapunov function can be potentially generalized to other GTD algorithms. Finally, we conduct experiments to verify the effectiveness of our $\mathtt{GES}(\lambda)$.

* This submission has been accepted by AAAI2021 
Viaarxiv icon

Sample Complexity of Policy Gradient Finding Second-Order Stationary Points

Dec 02, 2020
Long Yang, Qian Zheng, Gang Pan

Figure 1 for Sample Complexity of Policy Gradient Finding Second-Order Stationary Points
Figure 2 for Sample Complexity of Policy Gradient Finding Second-Order Stationary Points

The goal of policy-based reinforcement learning (RL) is to search the maximal point of its objective. However, due to the inherent non-concavity of its objective, convergence to a first-order stationary point (FOSP) can not guarantee the policy gradient methods finding a maximal point. A FOSP can be a minimal or even a saddle point, which is undesirable for RL. Fortunately, if all the saddle points are \emph{strict}, all the second-order stationary points (SOSP) are exactly equivalent to local maxima. Instead of FOSP, we consider SOSP as the convergence criteria to character the sample complexity of policy gradient. Our result shows that policy gradient converges to an $(\epsilon,\sqrt{\epsilon\chi})$-SOSP with probability at least $1-\widetilde{\mathcal{O}}(\delta)$ after the total cost of $\mathcal{O}\left(\dfrac{\epsilon^{-\frac{9}{2}}}{(1-\gamma)\sqrt\chi}\log\dfrac{1}{\delta}\right)$, where $\gamma\in(0,1)$. Our result improves the state-of-the-art result significantly where it requires $\mathcal{O}\left(\dfrac{\epsilon^{-9}\chi^{\frac{3}{2}}}{\delta}\log\dfrac{1}{\epsilon\chi}\right)$. Our analysis is based on the key idea that decomposes the parameter space $\mathbb{R}^p$ into three non-intersected regions: non-stationary point, saddle point, and local optimal region, then making a local improvement of the objective of RL in each region. This technique can be potentially generalized to extensive policy gradient methods.

* This submission has been accepted by AAAI2021 
Viaarxiv icon

Object Properties Inferring from and Transfer for Human Interaction Motions

Aug 20, 2020
Qian Zheng, Weikai Wu, Hanting Pan, Niloy Mitra, Daniel Cohen-Or, Hui Huang

Figure 1 for Object Properties Inferring from and Transfer for Human Interaction Motions
Figure 2 for Object Properties Inferring from and Transfer for Human Interaction Motions
Figure 3 for Object Properties Inferring from and Transfer for Human Interaction Motions
Figure 4 for Object Properties Inferring from and Transfer for Human Interaction Motions

Humans regularly interact with their surrounding objects. Such interactions often result in strongly correlated motion between humans and the interacting objects. We thus ask: "Is it possible to infer object properties from skeletal motion alone, even without seeing the interacting object itself?" In this paper, we present a fine-grained action recognition method that learns to infer such latent object properties from human interaction motion alone. This inference allows us to disentangle the motion from the object property and transfer object properties to a given motion. We collected a large number of videos and 3D skeletal motions of the performing actors using an inertial motion capture device. We analyze similar actions and learn subtle differences among them to reveal latent properties of the interacting objects. In particular, we learn to identify the interacting object, by estimating its weight, or its fragility or delicacy. Our results clearly demonstrate that the interaction motions and interacting objects are highly correlated and indeed relative object latent properties can be inferred from the 3D skeleton sequences alone, leading to new synthesis possibilities for human interaction motions. Dataset will be available at http://vcc.szu.edu.cn/research/2020/IT.

Viaarxiv icon