We propose a novel framework for integrating Hawkes processes with multi-armed bandit algorithms to solve spatio-temporal event forecasting and detection problems when data may be undersampled or spatially biased. In particular, we introduce an upper confidence bound algorithm using Bayesian spatial Hawkes process estimation for balancing the tradeoff between exploiting geographic regions where data has been collected and exploring geographic regions where data is unobserved. We first validate our model using simulated data and then apply it to the problem of disaster search and rescue using calls for service data from hurricane Harvey in 2017. Our model outperforms the state of the art baseline spatial MAB algorithms in terms of cumulative reward and several other ranking evaluation metrics.