Markov State--Space Modeling and Channel Characterization for DNA-Based Molecular Communication

In this paper, we study DNA-based molecular communication with microarray-style reception under reversible hybridization, where the bound-state observation exhibits both inter-symbol interference and colored counting noise. To capture these effects in a communication-oriented form, we develop a Markov state-space framework based on a voxelized reaction--diffusion model, in which a block-structured transition matrix describes molecular transport and binding/unbinding dynamics. For the microarray specialization, this representation yields the channel impulse response, the equilibrium gain, and a settling-time-based characterization of the effective channel memory. Building on the resulting symbol-rate observation model for on--off keying, we derive a grouped-binomial counting model and obtain a closed-form expression for the covariance of the counting noise. Based on these statistics, we further develop a differential-threshold detector and a finite-memory decision-feedback equalizer. Numerical results validate the theoretical correlation behavior and show that the relative performance of the proposed receivers depends strongly on the channel-memory regime.

Modulation, ISI, and Detection for Langmuir Adsorption-Based Microfluidic Molecular Communication

This paper studies microfluidic molecular communication receivers with finite-capacity Langmuir adsorption driven by an effective surface concentration. In the reaction-limited regime, we derive a closed-form single-pulse response kernel and a symbol-rate recursion for on-off keying that explicitly exposes channel memory and inter-symbol interference. We further develop short-pulse and long-pulse approximations, revealing an interference asymmetry in the long-pulse regime due to saturation. To account for stochasticity, we adopt a finite-receptor binomial counting model, employ pulse-end sampling, and propose a low-complexity midpoint-threshold detector that reduces to a fixed threshold when interference is negligible. Numerical results corroborate the proposed characterization and quantify detection performance versus pulse and symbol durations.

Selective Reviews of Bandit Problems in AI via a Statistical View

Reinforcement Learning (RL) is a widely researched area in artificial intelligence that focuses on teaching agents decision-making through interactions with their environment. A key subset includes stochastic multi-armed bandit (MAB) and continuum-armed bandit (SCAB) problems, which model sequential decision-making under uncertainty. This review outlines the foundational models and assumptions of bandit problems, explores non-asymptotic theoretical tools like concentration inequalities and minimax regret bounds, and compares frequentist and Bayesian algorithms for managing exploration-exploitation trade-offs. We also extend the discussion to $K$-armed contextual bandits and SCAB, examining their methodologies, regret analyses, and discussing the relation between the SCAB problems and the functional data analysis. Finally, we highlight recent advances and ongoing challenges in the field.