Eye gaze analysis is an important research problem in the field of computer vision and Human-Computer Interaction (HCI). Even with significant progress in the last few years, automatic gaze analysis still remains challenging due to the individuality of eyes, eye-head interplay, occlusion, image quality, and illumination conditions. There are several open questions including what are the important cues to interpret gaze direction in an unconstrained environment without prior knowledge and how to encode them in real-time. We review the progress across a range of gaze analysis tasks and applications to shed light on these fundamental questions; identify effective methods in gaze analysis and provide possible future directions. We analyze recent gaze estimation and segmentation methods, especially in the unsupervised and weakly supervised domain, based on their advantages and reported evaluation metrics. Our analysis shows that the development of a robust and generic gaze analysis method still needs to address real-world challenges such as unconstrained setup and learning with less supervision. We conclude by discussing future research directions for designing a real-world gaze analysis system that can propagate to other domains including computer vision, AR (Augmented Reality), VR (Virtual Reality), and HCI (Human Computer Interaction).