Unmanned Aerial Vehicle (UAV) has already demonstrated its potential in many civilian applications, and the fa\c{c}ade inspection is among the most promising ones. In this paper, we focus on enabling the autonomous perception and control of a small UAV for a fa\c{c}ade inspection task. Specifically, we consider the perception as a planar object pose estimation problem by simplifying the building structure as concatenation of planes, and the control as an optimal reference tracking control problem. First, a vision based adaptive observer is proposed which can realize stable plane pose estimation under very mild observation conditions. Second, a model predictive controller is designed to achieve stable tracking and smooth transition in a multi-plane scenario, while the persistent excitation (PE) condition of the observer and the maneuver constraints of the UAV are satisfied. The proposed autonomous plane pose estimation and plane tracking methods are tested in both simulation and practical building fas\c{c}ade inspection scenarios, which demonstrate their effectiveness and practicability.