The prevalence of Diabetes mellitus (DM) in the Middle East is exceptionally high as compared to the rest of the world. In fact, the prevalence of diabetes in the Middle East is 17-20%, which is well above the global average of 8-9%. Research has shown that food intake has strong connections with the blood glucose levels of a patient. In this regard, there is a need to build automatic tools to monitor the blood glucose levels of diabetics and their daily food intake. This paper presents an automatic way of tracking continuous glucose and food intake of diabetics using off-the-shelf sensors and machine learning, respectively. Our system not only helps diabetics to track their daily food intake but also assists doctors to analyze the impact of the food in-take on blood glucose in real-time. For food recognition, we collected a large-scale Middle-Eastern food dataset and proposed a fusion-based framework incorporating several existing pre-trained deep models for Middle-Eastern food recognition.