Actor-Critic Pretraining for Proximal Policy Optimization

Reinforcement learning (RL) actor-critic algorithms enable autonomous learning but often require a large number of environment interactions, which limits their applicability in robotics. Leveraging expert data can reduce the number of required environment interactions. A common approach is actor pretraining, where the actor network is initialized via behavioral cloning on expert demonstrations and subsequently fine-tuned with RL. In contrast, the initialization of the critic network has received little attention, despite its central role in policy optimization. This paper proposes a pretraining approach for actor-critic algorithms like Proximal Policy Optimization (PPO) that uses expert demonstrations to initialize both networks. The actor is pretrained via behavioral cloning, while the critic is pretrained using returns obtained from rollouts of the pretrained policy. The approach is evaluated on 15 simulated robotic manipulation and locomotion tasks. Experimental results show that actor-critic pretraining improves sample efficiency by 86.1% on average compared to no pretraining and by 30.9% to actor-only pretraining.

Behavioral Cloning for Robotic Connector Assembly: An Empirical Study

Automating the assembly of wire harnesses is challenging in automotive, electrical cabinet, and aircraft production, particularly due to deformable cables and a high variance in connector geometries. In addition, connectors must be inserted with limited force to avoid damage, while their poses can vary significantly. While humans can do this task intuitively by combining visual and haptic feedback, programming an industrial robot for such a task in an adaptable manner remains difficult. This work presents an empirical study investigating the suitability of behavioral cloning for learning an action prediction model for connector insertion that fuses force-torque sensing with a fixed position camera. We compare several network architectures and other design choices using a dataset of up to 300 successful human demonstrations collected via teleoperation of a UR5e robot with a SpaceMouse under varying connector poses. The resulting system is then evaluated against five different connector geometries under varying connector poses, achieving an overall insertion success rate of over 90 %.

The Biosensor based on electrochemical dynamics of fermentation in yeast Saccharomyces Cerevisiae

The zymase activity of the yeast Saccharomyces Cerevisiae is sensitive to environmental parameters and is therefore used as a microbiological sensor for water quality assessment, ecotoxicological characterization or environmental monitoring. Comparing to bacterial bioluminescence approach, this method has no toxicity, excludes usage of genetically modified microorganisms, and enables low-cost express analysis. This work focuses on measuring the yeast fermentation dynamics based on multichannel pressure sensing and electrochemical impedance spectroscopy (EIS). Measurement results are compared with each other in terms of accuracy, reproducibility and ease of use in the field conditions. It has been shown that EIS provides more information about ionic dynamics of metabolic processes and requires less complex measurements. The conducted experiments demonstrated the sensitivity of this approach for assessing biophotonic phenomena, non-chemical water treatments and impact of environmental stressors.