We consider the problem of learning predictive models from longitudinal data, consisting of irregularly repeated, sparse observations from a set of individuals over time. Effective approaches to this problem have to account for the complex multi-level correlation structure in the data. Gaussian process models offer an attractive framework for longitudinal data analysis (LDA) because they require fewer assumptions about the underlying data distribution compared to parametric models and model complex correlation structure using a composition of kernels. However, such methods have two key shortcomings: (i) The kernel often relies on ad hoc heuristics or a tedious process of trial and error; and (ii) The methods do not scale with increasing number of individuals, observations per individual, or the number of covariates. We present L-DKGPR, an effective and scalable longitudinal deep kernel Gaussian process regression model that overcomes the key limitations of existing GP based approaches to predictive modeling from longitudinal data. Specifically, L-DKGPR eliminates the need for trial and error or ad hoc heuristics in choosing a kernel function using a deep kernel learning technique which combines the advantages of modern deep neural networks (DNN) with the non-parametric flexibility of kernel methods, to automate the discovery of the rich correlation structure from the data. L-DKGPR adopts a multilevel model to account for the time-invariant individual-specific random effects and the time-varying fixed effects. We show how L-DKGPR can be efficiently trained using a variant of the stochastic variational method. We report the results of extensive experiments using both simulated and real-world benchmark longitudinal data sets that demonstrate the superior performance of L-DKGPR over the state-of-the-art methods.