Social media platforms such as Twitter provide an excellent resource for mobile communication during emergency events. During the sudden onset of a natural or artificial disaster, important information may be posted on Twitter or similar web forums. This information can be used for disaster response and crisis management if processed accurately. However, the data present in such situations is ever-changing, and considerable resources during such crisis may not be readily available. Therefore, a low resource, continually learning system must be developed to incorporate and make NLP models robust against noisy and unordered data. We utilise regularisation to alleviate catastrophic forgetting in the target neural networks while taking a distributed approach to enable learning on resource-constrained devices. We employ federated learning for distributed learning and aggregation of the central model for continual deployment.