The security issue of mobile robots have attracted considerable attention in recent years. Most existing works focus on detection and countermeasures for some classic attacks from cyberspace. Nevertheless, those work are generally based on some prior assumptions for the attacker (e.g., the system dynamics is known, or internal access is compromised). A few work are delicated to physical attacks, however, there still lacks certain intelligence and advanced control design. In this paper, we propose a physical-based and intelligent attack framework against the obstacle-avoidance of mobile robots. The novelty of our work lies in the following: i) Without any prior information of the system dynamics, the attacker can learn the detection area and goal position of a mobile robot by trial and observation, and the obstacle-avoidance mechanism is learned by support vector regression (SVR) method; ii) Considering different attack requirements, different attack strategies are proposed to implement the attack efficiently; iii) The framework is suitable for holonomic and non-holonomic mobile robots, and the algorithm performance analysis about time complexity and optimality is provided. Furthermore, the condition is obtained to guarantee the success of the attack. Simulations illustrate the effectiveness of the proposed framework.