This paper presents ongoing research in the SWARMs project towards facilitating context awareness in underwater robots. In particular, the focus of this paper is put on the context reasoning part. The underwater environment introduces uncertainties in context data which lead to difficulties in the context reasoning phase. As probability is the best well-known formalism for computational scientific reasoning under uncertainties, the emerging and effective probabilistic reasoning method, namely, Multi-Entity Bayesian Network (MEBN), is explored for its feasibility to reason under uncertainties in the SWARMs project. A simple use case for oil spill monitoring is used to verify the usefulness of MEBN. The results show that the MEBN is a promising approach to reason about context in the presence of uncertainties in the underwater robot field.