An omnidirectional multirotor has the advantageous maneuverability of decoupled translational and rotational motions, drastically superseding the traditional multirotors' motion capability. Such maneuverability requires an omnidirectional multirotor to frequently alter the thrust amplitude and even direction, which is prone to the rotors' settling time induced from the rotors' own dynamics. Furthermore, the omnidirectional multirotor's stability for tracking control in the presence of rotor dynamics has not yet been addressed. To resolve this issue, we propose a geometric tracking controller that takes the rotor dynamics into account. We show that the proposed controller yields the zero equilibrium of the error dynamics almost globally exponentially stable. The controller's tracking performance and stability are verified in simulations. Furthermore, the single-axis force experiment with the omnidirectional multirotor has been performed to confirm the proposed controller's performance in mitigating the rotors' settling time in the real world.