In this paper, we study a multiple-input multiple-output (MIMO) integrated sensing and communication (ISAC) system where one multi-antenna base station (BS) sends information to a user with multiple antennas in the downlink and simultaneously senses the location parameter of a target based on its reflected echo signals received back at the BS receive antennas. We focus on the case where the location parameter to be sensed is unknown and random, for which the prior distribution information is available for exploitation. First, we propose to adopt the posterior Cram\'er-Rao bound (PCRB) as the sensing performance metric with prior information, which quantifies a lower bound of the mean-squared error (MSE). Since the PCRB is in a complicated form, we derive a tight upper bound of it to draw more insights. Based on this, we analytically show that by exploiting the prior distribution information, the PCRB is always no larger than the CRB averaged over random location realizations without prior information exploitation. Next, we formulate the transmit covariance matrix optimization problem to minimize the sensing PCRB under a communication rate constraint. We obtain the optimal solution and derive useful properties on its rank. Then, by considering the derived PCRB upper bound as the objective function, we propose a low-complexity suboptimal solution in semi-closed form. Numerical results demonstrate the effectiveness of our proposed designs in MIMO ISAC exploiting prior information.