In this paper, we develop SecureD-FL, a privacy-preserving decentralized federated learning algorithm, i.e., without the traditional centralized aggregation server. For the decentralized aggregation, we employ the Alternating Direction Method of Multiplier (ADMM) and examine its privacy weakness. To address the privacy risk, we introduce a communication pattern inspired by the combinatorial block design theory and establish its theoretical privacy guarantee. We also propose an efficient algorithm to construct such a communication pattern. We evaluate our method on image classification and next-word prediction applications over federated benchmark datasets with nine and fifteen distributed sites hosting training data. While preserving privacy, SecureD-FL performs comparably to the standard centralized federated learning method; the degradation in test accuracy is only up to 0.73%.