Multimodal Signal Restoration with Signed Twofold Graph Learning

Multimodal signals on sensor networks are commonly modeled under the twofold graph assumption (TGA), which represents spatial structure and inter-modality relations as two separate graphs. Existing TGA-based signal restoration methods, however, either assume the graphs are known or restrict edge weights to be non-negative, preventing them from capturing negative inter-modal correlations. We address both limitations by formulating joint signal restoration and twofold graph learning as MAP estimation under a matrix normal prior, where the spatial and modality graph Laplacians appear directly as precision matrices. The resulting non-convex objective is solved by alternating minimization: The signal is updated via conjugate gradient applied to the arising Sylvester-type linear system; the graphs are updated via primal-dual hybrid gradient (PDHG). We further propose a method to estimate the signed structure of the modality graph from the dominant eigenspace of a complementary kernel matrix, which is then used in PDHG to update edge magnitudes. These iterative solvers are then unrolled into a feedforward network, with regularization weights and step sizes as layer-wise trainable parameters. Experiments on synthetic multimodal graph signals and a real Japan meteorological dataset confirm that the proposed method outperforms existing baselines across a range of noise levels and missing-data patterns.

Joint Alignment and Denoising for Event-Based Vision Sensors Using Regret-based Pareto Optimization

This paper proposes a joint alignment and denoising method for event-based vision sensors (EVSs). Existing signal processing methods for EVSs typically perform event alignment (EA) and event denoising (ED) as separate modules. However, this separation creates a dilemma: without ED, EA is biased by noise, whereas without EA, ED struggles to distinguish signal events from noise ones. To address this dilemma, we jointly optimize EA and ED by formulating a bi-objective Pareto optimization problem. Our formulation is built upon a contrast map that counts the number of events localized in each pixel. With a contrast map, we can formulate EA as maximizing its variance and ED as minimizing the variance. We cast these two conflicting problems as a Pareto optimization and use a regret strategy to obtain a solution. Experimental results on denoising and motion estimation demonstrate that our method achieves improvements against alternative ones.

Denoising for Neuromorphic Cameras Based on Graph Spectral Features

Neuromorphic cameras, also known as event-based cameras, can detect changes in the environmental brightness asynchronously and independently for each pixel. They output the brightness changes, i.e., events, as 3-D (2-D pixel coordinates + time) streaming data. While event-based cameras are used in many applications because of their desirable characteristics, e.g., high temporal resolution, low latency, low power consumption, and high dynamic range, their measurements contain considerable noise due to their high sensitivity. In this paper, we propose a denoising method for event-based cameras based on graph spectral features. In the proposed method, we first construct a graph where nodes represent events and edges represent the spatiotemporal distance between the events. To calculate the graph-specified parameter that controls the connectivities of a constructed graph, we utilize the prior on the density of 3-D events. We then calculate the eigenvectors of the graph Laplacian. The obtained eigenvectors are used to extract noiseless events directly. In the calculation of the eigenvectors, we customize the graph Laplacian to reorder its eigenvalues. This allows us to leverage fast eigensolver algorithms instead of the naive eigendecomposition and thereby reduce computational complexity. In experiments on synthetic and real-world event data, we demonstrate that the proposed method effectively removes noise events from the raw events compared to alternative methods.

Dynamic Sensor Scheduling Based on Node Partitioning of Graphs

This paper proposes a dynamic sensor scheduling method for sensor networks. In sensor network applications, we often need multiple equally-informative node subsets that are activated sequentially to make a sensor network robust against concentrated battery consumption and sensor failures. In addition, quality of these subsets changes dynamically and thus we must adapt those changes. To find those node subsets, we propose a graph node partitioning method based on sampling theory for graph signals. We aim to minimize the average reconstruction error for signals obtained at all node subsets, in contrast to conventional single subset selection. The graph node partitioning problem is formulated as a difference-of-convex (DC) optimization based on a subspace prior of graph signals, and is solved by the proximal DC algorithm. It guarantees convergence to a critical point. To accommodate the online scenario where the signal subspace and optimal partitioning may change over time, we adaptively estimate the signal subspace from historical data and sequentially update the prior for our partitioning method. Numerical experiments on synthetic and real-world sensor network data demonstrate that the proposed method achieves lower average mean squared errors compared to alternative methods.

Auditory Attention Decoding without Spatial Information: A Diotic EEG Study

Auditory attention decoding (AAD) identifies the attended speech stream in multi-speaker environments by decoding brain signals such as electroencephalography (EEG). This technology is essential for realizing smart hearing aids that address the cocktail party problem and for facilitating objective audiometry systems. Existing AAD research mainly utilizes dichotic environments where different speech signals are presented to the left and right ears, enabling models to classify directional attention rather than speech content. However, this spatial reliance limits applicability to real-world scenarios, such as the "cocktail party" situation, where speakers overlap or move dynamically. To address this challenge, we propose an AAD framework for diotic environments where identical speech mixtures are presented to both ears, eliminating spatial cues. Our approach maps EEG and speech signals into a shared latent space using independent encoders. We extract speech features using wav2vec 2.0 and encode them with a 2-layer 1D convolutional neural network (CNN), while employing the BrainNetwork architecture for EEG encoding. The model identifies the attended speech by calculating the cosine similarity between EEG and speech representations. We evaluate our method on a diotic EEG dataset and achieve 72.70% accuracy, which is 22.58% higher than the state-of-the-art direction-based AAD method.

Graph Signal Denoising Using Regularization by Denoising and Its Parameter Estimation

In this paper, we propose an interpretable denoising method for graph signals using regularization by denoising (RED). RED is a technique developed for image restoration that uses an efficient (and sometimes black-box) denoiser in the regularization term of the optimization problem. By using RED, optimization problems can be designed with the explicit use of the denoiser, and the gradient of the regularization term can be easily computed under mild conditions. We adapt RED for denoising of graph signals beyond image processing. We show that many graph signal denoisers, including graph neural networks, theoretically or practically satisfy the conditions for RED. We also study the effectiveness of RED from a graph filter perspective. Furthermore, we propose supervised and unsupervised parameter estimation methods based on deep algorithm unrolling. These methods aim to enhance the algorithm applicability, particularly in the unsupervised setting. Denoising experiments for synthetic and real-world datasets show that our proposed method improves signal denoising accuracy in mean squared error compared to existing graph signal denoising methods.

Algorithm Unrolling-based Denoising of Multimodal Graph Signals

We propose a denoising method of multimodal graph signals by iteratively solving signal restoration and graph learning problems. Many complex-structured data, i.e., those on sensor networks, can capture multiple modalities at each measurement point, referred to as modalities. They are also assumed to have an underlying structure or correlations in modality as well as space. Such multimodal data are regarded as graph signals on a twofold graph and they are often corrupted by noise. Furthermore, their spatial/modality relationships are not always given a priori: We need to estimate twofold graphs during a denoising algorithm. In this paper, we consider a signal denoising method on twofold graphs, where graphs are learned simultaneously. We formulate an optimization problem for that and parameters in an iterative algorithm are learned from training data by unrolling the iteration with deep algorithm unrolling. Experimental results on synthetic and real-world data demonstrate that the proposed method outperforms existing model- and deep learning-based graph signal denoising methods.

Joint Graph Estimation and Signal Restoration for Robust Federated Learning

We propose a robust aggregation method for model parameters in federated learning (FL) under noisy communications. FL is a distributed machine learning paradigm in which a central server aggregates local model parameters from multiple clients. These parameters are often noisy and/or have missing values during data collection, training, and communication between the clients and server. This may cause a considerable drop in model accuracy. To address this issue, we learn a graph that represents pairwise relationships between model parameters of the clients during aggregation. We realize it with a joint problem of graph learning and signal (i.e., model parameters) restoration. The problem is formulated as a difference-of-convex (DC) optimization, which is efficiently solved via a proximal DC algorithm. Experimental results on MNIST and CIFAR-10 datasets show that the proposed method outperforms existing approaches by up to $2$--$5\%$ in classification accuracy under biased data distributions and noisy conditions.

Multiscale Graph Construction Using Non-local Cluster Features

This paper presents a multiscale graph construction method using both graph and signal features. Multiscale graph is a hierarchical representation of the graph, where a node at each level indicates a cluster in a finer resolution. To obtain the hierarchical clusters, existing methods often use graph clustering; however, they may ignore signal variations. As a result, these methods could fail to detect the clusters having similar features on nodes. In this paper, we consider graph and node-wise features simultaneously for multiscale clustering of a graph. With given clusters of the graph, the clusters are merged hierarchically in three steps: 1) Feature vectors in the clusters are extracted. 2) Similarities among cluster features are calculated using optimal transport. 3) A variable $k$-nearest neighbor graph (V$k$NNG) is constructed and graph spectral clustering is applied to the V$k$NNG to obtain clusters at a coarser scale. Additionally, the multiscale graph in this paper has \textit{non-local} characteristics: Nodes with similar features are merged even if they are spatially separated. In experiments on multiscale image and point cloud segmentation, we demonstrate the effectiveness of the proposed method.

Time-Varying Graph Signal Estimation among Multiple Sub-Networks

This paper presents an estimation method for time-varying graph signals among multiple sub-networks. In many sensor networks, signals observed are associated with nodes (i.e., sensors), and edges of the network represent the inter-node connectivity. For a large sensor network, measuring signal values at all nodes over time requires huge resources, particularly in terms of energy consumption. To alleviate the issue, we consider a scenario that a sub-network, i.e., cluster, from the whole network is extracted and an intra-cluster analysis is performed based on the statistics in the cluster. The statistics are then utilized to estimate signal values in another cluster. This leads to the requirement for transferring a set of parameters of the sub-network to the others, while the numbers of nodes in the clusters are typically different. In this paper, we propose a cooperative Kalman filter between two sub-networks. The proposed method alternately estimates signals in time between two sub-networks. We formulate a state-space model in the source cluster and transfer it to the target cluster on the basis of optimal transport. In the signal estimation experiments of synthetic and real-world signals, we validate the effectiveness of the proposed method.