MSI-NeRF: Linking Omni-Depth with View Synthesis through Multi-Sphere Image aided Generalizable Neural Radiance Field

Panoramic observation using fisheye cameras is significant in robot perception, reconstruction, and remote operation. However, panoramic images synthesized by traditional methods lack depth information and can only provide three degrees-of-freedom (3DoF) rotation rendering in virtual reality applications. To fully preserve and exploit the parallax information within the original fisheye cameras, we introduce MSI-NeRF, which combines deep learning omnidirectional depth estimation and novel view rendering. We first construct a multi-sphere image as a cost volume through feature extraction and warping of the input images. It is then processed by geometry and appearance decoders, respectively. Unlike methods that regress depth maps directly, we further build an implicit radiance field using spatial points and interpolated 3D feature vectors as input. In this way, we can simultaneously realize omnidirectional depth estimation and 6DoF view synthesis. Our method is trained in a semi-self-supervised manner. It does not require target view images and only uses depth data for supervision. Our network has the generalization ability to reconstruct unknown scenes efficiently using only four images. Experimental results show that our method outperforms existing methods in depth estimation and novel view synthesis tasks.

One Size Does not Fit All: Personalised Affordance Design for Social Robots

Personalisation is essential to achieve more acceptable and effective results in human-robot interaction. Placing users in the central role, many studies have focused on enhancing the abilities of social robots to perceive and understand users. However, little is known about improving user perceptions and interpretation of a social robot in spoken interactions. The work described in the paper aims to find out what affects the personalisation of affordance of a social robot, namely its appearance, voice and language behaviours. The experimental data presented here is based on an ongoing project. It demonstrates the many and varied ways in which people change their preferences for the affordance of a social robot under different circumstances. It also examines the relationship between such preferences and expectations of characteristics of a social robot, like competence and warmth. It also shows that individuals have different perceptions of the language behaviours of the same robot. These results demonstrate that one-sized personalisation does not fit all. Personalisation should be considered a comprehensive approach, including appropriate affordance design, to suit the user expectations of social roles.