We consider core-periphery structured graphs, which are graphs with a group of densely and sparsely connected nodes, respectively, referred to as core and periphery nodes. The so-called core score of a node is related to the likelihood of it being a core node. In this paper, we focus on learning the core scores of a graph from its node attributes and connectivity structure. To this end, we propose two classes of probabilistic graphical models: affine and nonlinear. First, we describe affine generative models to model the dependence of node attributes on its core scores, which determine the graph structure. Next, we discuss nonlinear generative models in which the partial correlations of node attributes influence the graph structure through latent core scores. We develop algorithms for inferring the model parameters and core scores of a graph when both the graph structure and node attributes are available. When only the node attributes of graphs are available, we jointly learn a core-periphery structured graph and its core scores. We provide results from numerical experiments on several synthetic and real-world datasets to demonstrate the efficacy of the developed models and algorithms.