Nonlinear Processing with Linear Optics

Deep neural networks have achieved remarkable breakthroughs by leveraging multiple layers of data processing to extract hidden representations, albeit at the cost of large electronic computing power. To enhance energy efficiency and speed, the optical implementation of neural networks aims to harness the advantages of optical bandwidth and the energy efficiency of optical interconnections. In the absence of low-power optical nonlinearities, the challenge in the implementation of multilayer optical networks lies in realizing multiple optical layers without resorting to electronic components. In this study, we present a novel framework that uses multiple scattering that is capable of synthesizing programmable linear and nonlinear transformations concurrently at low optical power by leveraging the nonlinear relationship between the scattering potential, represented by data, and the scattered field. Theoretical and experimental investigations show that repeating the data by multiple scattering enables non-linear optical computing at low power continuous wave light.

Forward-Forward Training of an Optical Neural Network

Neural networks (NN) have demonstrated remarkable capabilities in various tasks, but their computation-intensive nature demands faster and more energy-efficient hardware implementations. Optics-based platforms, using technologies such as silicon photonics and spatial light modulators, offer promising avenues for achieving this goal. However, training multiple trainable layers in tandem with these physical systems poses challenges, as they are difficult to fully characterize and describe with differentiable functions, hindering the use of error backpropagation algorithm. The recently introduced Forward-Forward Algorithm (FFA) eliminates the need for perfect characterization of the learning system and shows promise for efficient training with large numbers of programmable parameters. The FFA does not require backpropagating an error signal to update the weights, rather the weights are updated by only sending information in one direction. The local loss function for each set of trainable weights enables low-power analog hardware implementations without resorting to metaheuristic algorithms or reinforcement learning. In this paper, we present an experiment utilizing multimode nonlinear wave propagation in an optical fiber demonstrating the feasibility of the FFA approach using an optical system. The results show that incorporating optical transforms in multilayer NN architectures trained with the FFA, can lead to performance improvements, even with a relatively small number of trainable weights. The proposed method offers a new path to the challenge of training optical NNs and provides insights into leveraging physical transformations for enhancing NN performance.

Nonlinear Optical Data Transformer for Machine Learning

Modern machine learning models use an ever-increasing number of parameters to train (175 billion parameters for GPT-3) with large datasets to obtain better performance. Bigger is better has been the norm. Optical computing has been reawakened as a potential solution to large-scale computing through optical accelerators that carry out linear operations while reducing electrical power. However, to achieve efficient computing with light, creating and controlling nonlinearity optically rather than electronically remains a challenge. This study explores a reservoir computing (RC) approach whereby a 14 mm long few-mode waveguide in LiNbO3 on insulator is used as a complex nonlinear optical processor. A dataset is encoded digitally on the spectrum of a femtosecond pulse which is then launched in the waveguide. The output spectrum depends nonlinearly on the input. We experimentally show that a simple digital linear classifier with 784 parameters using the output spectrum from the waveguide as input increased the classification accuracy of several databases compared to non-transformed data, approximately 10$\%$. In comparison, a deep digital neural network (NN) with 40000 parameters was necessary to achieve the same accuracy. Reducing the number of parameters by a factor of $\sim$50 illustrates that a compact optical RC approach can perform on par with a deep digital NN.

Variational framework for partially-measured physical system control: examples of vision neuroscience and optical random media

To characterize a physical system to behave as desired, either its underlying governing rules must be known a priori or the system itself be accurately measured. The complexity of full measurements of the system scales with its size. When exposed to real-world conditions, such as perturbations or time-varying settings, the system calibrated for a fixed working condition might require non-trivial re-calibration, a process that could be prohibitively expensive, inefficient and impractical for real-world use cases. In this work, we propose a learning procedure to obtain a desired target output from a physical system. We use Variational Auto-Encoders (VAE) to provide a generative model of the system function and use this model to obtain the required input of the system that produces the target output. We showcase the applicability of our method for two datasets in optical physics and neuroscience.

Scalable Optical Learning Operator

Today's heavy machine learning tasks are fueled by large datasets. Computing is performed with power hungry processors whose performance is ultimately limited by the data transfer to and from memory. Optics is one of the powerful means of communicating and processing information and there is intense current interest in optical information processing for realizing high-speed computations. Here we present and experimentally demonstrate an optical computing framework based on spatiotemporal effects in multimode fibers for a range of learning tasks from classifying COVID-19 X-ray lung images and speech recognition to predicting age from face images. The presented framework overcomes the energy scaling problem of existing systems without compromising speed. We leveraged simultaneous, linear, and nonlinear interaction of spatial modes as a computation engine. We numerically and experimentally showed the ability of the method to execute several different tasks with accuracy comparable to a digital implementation. Our results indicate that a powerful supercomputer would be required to duplicate the performance of the multimode fiber-based computer.

Multimode Fiber Projector

Direct image transmission in multimode fibers (MMFs) is hampered by modal scrambling inside the fiber due to the multimodal nature of the medium. To undo modal scrambling, approaches that either use interferometry to construct a transmission matrix or iterative feedback based wavefront shaping to form an output spot on the camera have been proposed and implemented successfully. The former method entails measuring the complex output field (phase and amplitude) using interferometric systems. The latter, requires scanning the spot by phase conjugation or iterative techniques to form arbitrary shapes, increasing the computational cost. In this work, we show that by using neural networks, we are able to project arbitrary shapes through the MMF without measuring the output phase. Specifically, we demonstrate that our projector network is able to produce input patterns that, when sent through the fiber, form arbitrary shapes on the camera with fidelities (correlation) as high as ~90%. We believe this approach opens up new paths towards imaging and pattern projection for a plethora of applications ranging from tissue imaging, surgical ablations to virtual/augmented reality.