Visual Semantic Decoding of Electrocorticography from Video Stimuli using End-to-End Deep Learning

ECoG-based visual semantic decoding enables inference of semantic interpretation of visual perception from complex, noisy brain activity. This study examines the feasibility of visual semantic decoding using an end-to-end deep learning framework using electrocorticography (ECoG). Specifically, the decoding task is to predict visual categories from video stimuli using time-series neural inputs. A previously collected ECoG dataset from participants ($n=17$) with drug-resistant epilepsy is used for analysis. With fewer than 50 training samples per visual category, this study evaluates multiple deep learning approaches, artificial neural network architectures, and frequency-band filtered inputs. The best-performing approach is analyzed to shed light on the discriminative information it relies on across spectral, temporal, and cortical dimensions. The selected decoding system uses mixup augmentation, a Transformer-based encoder, and high-gamma (80-150 Hz) inputs with a 900 ms post-stimulus window. Further analysis shows that early visual cortex (V2-V4), ventral stream visual cortex, MT+ complex with neighbouring visual areas, and lateral temporal cortex contributed substantially to decoding performance. This study demonstrates that an end-to-end deep learning framework can yield promising decoding performance from dynamic visual stimuli without handcrafted features, while the model behavior remains interpretable through spectral, temporal, and cortical dimensions, which are broadly consistent with established neuroscience knowledge.

Sub-Scalp Brain-Computer Interface Device Design and Fabrication

Current brain-computer interfaces (BCI) face limitations in signal acquisition. While sub-scalp EEG offers a potential solution, existing devices prioritize chronic seizure monitoring and lack features suited for BCI applications. This work addresses this gap by outlining key specifications for sub-scalp BCI devices, focusing on channel count, sampling rate, power efficiency, and form factor. We present the Set-And-Forget EEG (SAFE) system, a custom-built amplifier and wireless transmitter meeting these criteria. This compact (12x12 mm), six-channel device offers 1024 Hz sampling and Bluetooth Low Energy data transmission. Validation using generated sinusoids and electrocorticography recordings of visual evoked potentials in sheep models demonstrated low noise recording. Future animal studies will assess sub-scalp EEG signal quality for BCI applications. This data lays the groundwork for human trials, ultimately paving the way for chronic, in-home BCIs that empower individuals with physical disabilities.

Comparison of Sub-Scalp EEG and Endovascular Stent-Electrode Array for Visual Evoked Potential Brain-Computer Interface

Brain-computer interfaces (BCI) have the potential to improve the quality of life for persons with paralysis. Sub-scalp EEG provides an alternative BCI signal acquisition method that compromises between the limitations of traditional EEG systems and the risks associated with intracranial electrodes, and has shown promise in long-term seizure monitoring. However, sub-scalp EEG has not yet been assessed for suitability in BCI applications. This study presents a preliminary comparison of visual evoked potentials (VEPs) recorded using subscalp and endovascular stent electrodes in a sheep. Sub-scalp electrodes recorded comparable VEP amplitude, signal-to-noise ratio and bandwidth to the stent electrodes.