This paper addresses the problem of real-time vision-based autonomous obstacle avoidance in unstructured environments for quadrotor UAVs. We assume that our UAV is equipped with a forward facing stereo camera as the only sensor to perceive the world around it. Moreover, all the computations are performed onboard. Feasible trajectory generation in this kind of problems requires rapid collision checks along with efficient planning algorithms. We propose a trajectory generation approach in the depth image space, which refers to the environment information as depicted by the depth images. In order to predict the collision in a look ahead robot trajectory, we create depth images from the sequence of robot poses along the path. We compare these images with the depth images of the actual world sensed through the forward facing stereo camera. We aim at generating fuel optimal trajectories inside the depth image space. In case of a predicted collision, a switching strategy is used to aggressively deviate the quadrotor away from the obstacle. For this purpose we use two closed loop motion primitives based on Linear Quadratic Regulator (LQR) objective functions. The proposed approach is validated through simulation and hardware experiments.