Awareness in robotics: An early perspective from the viewpoint of the EIC Pathfinder Challenge "Awareness Inside''

Consciousness has been historically a heavily debated topic in engineering, science, and philosophy. On the contrary, awareness had less success in raising the interest of scholars in the past. However, things are changing as more and more researchers are getting interested in answering questions concerning what awareness is and how it can be artificially generated. The landscape is rapidly evolving, with multiple voices and interpretations of the concept being conceived and techniques being developed. The goal of this paper is to summarize and discuss the ones among these voices connected with projects funded by the EIC Pathfinder Challenge called ``Awareness Inside'', a nonrecurring call for proposals within Horizon Europe designed specifically for fostering research on natural and synthetic awareness. In this perspective, we dedicate special attention to challenges and promises of applying synthetic awareness in robotics, as the development of mature techniques in this new field is expected to have a special impact on generating more capable and trustworthy embodied systems.

Unsupervised Statistical Feature-Guided Diffusion Model for Sensor-based Human Activity Recognition

Recognizing human activities from sensor data is a vital task in various domains, but obtaining diverse and labeled sensor data remains challenging and costly. In this paper, we propose an unsupervised statistical feature-guided diffusion model for sensor-based human activity recognition. The proposed method aims to generate synthetic time-series sensor data without relying on labeled data, addressing the scarcity and annotation difficulties associated with real-world sensor data. By conditioning the diffusion model on statistical information such as mean, standard deviation, Z-score, and skewness, we generate diverse and representative synthetic sensor data. We conducted experiments on public human activity recognition datasets and compared the proposed method to conventional oversampling methods and state-of-the-art generative adversarial network methods. The experimental results demonstrate that the proposed method can improve the performance of human activity recognition and outperform existing techniques.

User Localization using RF Sensing: A Performance comparison between LIS and mmWave Radars

Since electromagnetic signals are omnipresent, Radio Frequency (RF)-sensing has the potential to become a universal sensing mechanism with applications in localization, smart-home, retail, gesture recognition, intrusion detection, etc. Two emerging technologies in RF-sensing, namely sensing through Large Intelligent Surfaces (LISs) and mmWave Frequency-Modulated Continuous-Wave (FMCW) radars, have been successfully applied to a wide range of applications. In this work, we compare LIS and mmWave radars for localization in real-world and simulated environments. In our experiments, the mmWave radar achieves 0.71 Intersection Over Union (IOU) and 3cm error for bounding boxes, while LIS has 0.56 IOU and 10cm distance error. Although the radar outperforms the LIS in terms of accuracy, LIS features additional applications in communication in addition to sensing scenarios.

Energy-Efficient Design for RIS-assisted UAVcommunications in beyond-5G Networks

The usage of Reconfigurable Intelligent Surfaces (RIS) in conjunction with Unmanned Ariel Vehicles (UAVs) is being investigated as a way to provide energy-efficient communication to ground users in dense urban areas. In this paper, we devise an optimization scenario to reduce overall energy consumption in the network while guaranteeing certain Quality of Service (QoS) to the ground users in the area. Due to the complex nature of the optimization problem, we provide a joint UAV trajectory and RIS phase decision to minimize transmission power of the UAV and Base Station (BS) that yields good performance with lower complexity. So, the proposed method uses a Successive Convex Approximation (SCA) to iteratively determine a joint optimal solution for UAV Trajectory, RIS phase and BS and UAV Transmission Power. The approach has, therefore, been analytically evaluated under different sets of criterion.

Integrating Sensing and Communication in Cellular Networks via NR Sidelink

RF-sensing, the analysis and interpretation of movement or environment-induced patterns in received electromagnetic signals, has been actively investigated for more than a decade. Since electromagnetic signals, through cellular communication systems, are omnipresent, RF sensing has the potential to become a universal sensing mechanism with applications in smart home, retail, localization, gesture recognition, intrusion detection, etc. Specifically, existing cellular network installations might be dual-used for both communication and sensing. Such communications and sensing convergence is envisioned for future communication networks. We propose the use of NR-sidelink direct device-to-device communication to achieve device-initiated,flexible sensing capabilities in beyond 5G cellular communication systems. In this article, we specifically investigate a common issue related to sidelink-based RF-sensing, which is its angle and rotation dependence. In particular, we discuss transformations of mmWave point-cloud data which achieve rotational invariance, as well as distributed processing based on such rotational invariant inputs, at angle and distance diverse devices. To process the distributed data, we propose a graph based encoder to capture spatio-temporal features of the data and propose four approaches for multi-angle learning. The approaches are compared on a newly recorded and openly available dataset comprising 15 subjects, performing 21 gestures which are recorded from 8 angles.

Tesla-Rapture: A Lightweight Gesture Recognition System from mmWave Radar Point Clouds

We present Tesla-Rapture, a gesture recognition interface for point clouds generated by mmWave Radars. State of the art gesture recognition models are either too resource consuming or not sufficiently accurate for integration into real-life scenarios using wearable or constrained equipment such as IoT devices (e.g. Raspberry PI), XR hardware (e.g. HoloLens), or smart-phones. To tackle this issue, we developed Tesla, a Message Passing Neural Network (MPNN) graph convolution approach for mmWave radar point clouds. The model outperforms the state of the art on two datasets in terms of accuracy while reducing the computational complexity and, hence, the execution time. In particular, the approach, is able to predict a gesture almost 8 times faster than the most accurate competitor. Our performance evaluation in different scenarios (environments, angles, distances) shows that Tesla generalizes well and improves the accuracy up to 20% in challenging scenarios like a through-wall setting and sensing at extreme angles. Utilizing Tesla, we develop Tesla-Rapture, a real-time implementation using a mmWave Radar on a Raspberry PI 4 and evaluate its accuracy and time-complexity. We also publish the source code, the trained models, and the implementation of the model for embedded devices.

A cloud-IoT platform for passive radio sensing: challenges and application case studies

We propose a platform for the integration of passive radio sensing and vision technologies into a cloud-IoT framework that performs real-time channel quality information (CQI) time series processing and analytics. Radio sensing and vision technologies allow to passively detect and track objects or persons by using radio waves as probe signals that encode a 2D/3D view of the environment they propagate through. View reconstruction from the received radio signals, or CQI, is based on real-time data processing tools, that combine multiple radio measurements from possibly heterogeneous IoT networks. The proposed platform is designed to efficiently store and analyze CQI time series of different types and provides formal semantics for CQI data manipulation (ontology models). Post-processed data can be then accessible to third parties via JSON-REST calls. Finally, the proposed system supports the reconfiguration of CQI data collection based on the respective application. The performance of the proposed tools are evaluated through two experimental case studies that focus on assisted living applications in a smart-space environment and on driver behavior recognition for in-car control services. Both studies adopt and compare different CQI manipulation models and radio devices as supported by current and future (5G) standards.

A Multisensory Edge-Cloud Platform for Opportunistic Radio Sensing in Cobot Environments

Worker monitoring and protection in collaborative robot (cobots) industrial environments requires advanced sensing capabilities and flexible solutions to monitor the movements of the operator in close proximity of moving robots. Collaborative robotics is an active research area where Internet of Things (IoT) and novel sensing technologies are expected to play a critical role. Considering that no single technology can currently solve the problem of continuous worker monitoring, the paper targets the development of an IoT multisensor data fusion (MDF) platform. It is based on an edge-cloud architecture that supports the combination and transformation of multiple sensing technologies to enable the passive and anonymous detection of workers. Multidimensional data acquisition from different IoT sources, signal pre-processing, feature extraction, data distribution and fusion, along with machine learning (ML) and computing methods are described. The proposed IoT platform also comprises a practical solution for data fusion and analytics. It is able to perform opportunistic and real-time perception of workers by fusing and analyzing radio signals obtained from several interconnected IoT components, namely a multi-antenna WiFi installation (2.4-5 GHz), a sub-THz imaging camera (100 GHz), a network of radars (122 GHz) and infrared sensors (8-13 {\mu}m). The performance of the proposed IoT platform is validated through real use case scenarios inside a pilot industrial plant in which protective human--robot distance must be guaranteed considering latency and detection uncertainties.