Transportation electrification requires an increasing number of electric components (e.g., electric motors and electric energy storage systems) on vehicles, and control of the electric powertrains usually involves multiple inputs and multiple outputs (MIMO). This paper focused on the online optimization of energy management strategy for a multi-mode hybrid electric vehicle based on multi-agent reinforcement learning (MARL) algorithms that aim to address MIMO control optimization while most existing methods only deal with single output control. A new collaborative cyber-physical learning with multi-agents is proposed based on the analysis of the evolution of energy efficiency of the multi-mode hybrid electric vehicle (HEV) optimized by a deep deterministic policy gradient (DDPG)-based MARL algorithm. Then a learning driving cycle is set by a novel random method to speed up the training process. Eventually, network design, learning rate, and policy noise are incorporated in the sensibility analysis and the DDPG-based algorithm parameters are determined, and the learning performance with the different relationships of multi-agents is studied and demonstrates that the not completely independent relationship with Ratio 0.2 is the best. The compassion study with the single-agent and multi-agent suggests that the multi-agent can achieve approximately 4% improvement of total energy over the single-agent scheme. Therefore, the multi-objective control by MARL can achieve good optimization effects and application efficiency.