With the advantages of member diversity and team scale, heterogeneous multi-robot systems (HMRS) are widely used in complex scenarios, including disaster search and rescue, site surveillance, and traffic control. However, due to the variety of task requirements, it is still challenging to accurately allocate limited team capability to satisfy various task needs effectively. In this paper, a novel adaptive cooperation method, inner attention innerATT is developed to flexibly team heterogeneous robots to execute tasks as task needs change. innerATT is designed based on an attention mechanism and a multi-agent actor-critic reinforcement learning algorithm. We briefly validate how the inner attention mechanism can be exploited to enable flexible and robust decision making in guiding cooperation. The results, in two designed scenarios "task variety" and "robot availability variety", show that innerATT can enable flexible cooperation and reduce resource consumption in search and rescue tasks.