This paper explores a novel method for anxiety detection in older adults using simple wristband sensors such as Electrodermal Activity (EDA) and Photoplethysmogram (PPG) and a context-based feature. The proposed method for anxiety detection combines features from a single physiological signal with an experimental context-based feature to improve the performance of the anxiety detection model. The experimental data for this work is obtained from a year-long experiment on 41 healthy older adults (26 females and 15 males) in the age range 60-80 with mean age 73.36+-5.25 during a Trier Social Stress Test (TSST) protocol. The anxiety level ground truth was obtained from State-Trait Anxiety Inventory (STAI), which is regarded as the gold standard to measure perceived anxiety. EDA and Blood Volume Pulse (BVP) signals were recorded using a wrist-worn EDA and PPG sensor respectively. 47 features were computed from EDA and BVP signal, out of which a final set of 24 significantly correlated features were selected for analysis. The phases of the experimental study are encoded as unique integers to generate the context feature vector. A combination of features from a single sensor with the context feature vector is used for training a machine learning model to distinguish between anxious and not-anxious states. Results and analysis showed that the EDA and BVP machine learning models that combined the context feature along with the physiological features achieved 3.37% and 6.41% higher accuracy respectively than the models that used only physiological features. Further, end-to-end processing of EDA and BVP signals was simulated for real-time anxiety level detection. This work demonstrates the practicality of the proposed anxiety detection method in facilitating long-term monitoring of anxiety in older adults using low-cost consumer devices.