Fish schools present high-efficiency group behaviors through simple individual interactions to collective migration and dynamic escape from the predator. The school behavior of fish is usually a good inspiration to design control architecture for swarm robots. In this paper, a novel fish-inspired self-adaptive approach is proposed for collective escape for the swarm robots. In addition, a bio-inspired neural network (BINN) is introduced to generate collision-free escape robot trajectories through the combination of attractive and repulsive forces. Furthermore, to cope with dynamic environments, a neurodynamics-based self-adaptive mechanism is proposed to improve the self-adaptive performance of the swarm robots in the changing environment. Similar to fish escape maneuvers, simulation and experimental results show that the swarm robots are capable of collectively leaving away from the threats. Several comparison studies demonstrated that the proposed approach can significantly improve the effectiveness and efficiency of system performance, and the flexibility and robustness in complex environments.