The aim of many studies in biomedicine is to infer cause-effect relationships rather then simple associations. For instance, researcher might gather data on dietary habits, and cognitive function in an elderly population to determine which factors affect cognitive decline and to predict the effects of changes in diet on cognition. Finding answers to questions where we want to know the outcome after some other variable has been manipulated, is the subject of causal inference. Inferring causal effects from observational data is challenging and requires making untestable assumptions about the data-generating process. One essential assumption is that of no unmeasured confounder. In complex neuroimaging studies, we neither know all potential confounders nor do we have data on them. Thus, estimating a substitute confounder would be desirable. While this is infeasible in general, we will illustrate that by considering multiple causes of interest simultaneously, we can leverage the dependencies among them to identify causal effects by means of a latent factor model. In our experiments, we quantitatively evaluate the effectiveness of our approach on semi-synthetic data, where we know the true causal effects, and illustrate its use on real data on Alzheimer's disease, where it reveals important causes that otherwise would have been missed.