A Spiking Neural Network (SNN) trained with Spike Time Dependent Plasticity (STDP) is a neuro-inspired unsupervised learning method for various machine learning applications. This paper studies the generalizability properties of the STDP learning processes using the Hausdorff dimension of the trajectories of the learning algorithm. The paper analyzes the effects of STDP learning models and associated hyper-parameters on the generalizability properties of an SNN and characterizes the generalizability vs learnability trade-off in an SNN. The analysis is used to develop a Bayesian optimization approach to optimize the hyper-parameters for an STDP model to improve the generalizability properties of an SNN.