Set-based estimation has gained a lot of attention due to its ability to guarantee state enclosures for safety-critical systems. However, it requires computationally expensive operations, which in turn often requires outsourcing of these operations to cloud-computing platforms. Consequently, this raises some concerns with regard to sharing sensitive information and measurements. This paper presents the first privacy-preserving set-based estimation protocols using partially homomorphic encryption in which we preserve the privacy of the set of all possible estimates and the measurements. We consider a linear discrete-time dynamical system with bounded modeling and measurement uncertainties without any other statistical assumptions. We represent sets by zonotopes and constrained zonotopes as they can compactly represent high-dimensional sets and are closed under linear maps and Minkowski addition. By selectively encrypting some parameters of the used set representations, we are able to intersect sets in the encrypted domain, which enables guaranteed state estimation while ensuring the privacy goals. In particular, we show that our protocols achieve computational privacy using formal cryptographic definitions of computational indistinguishability. We demonstrate the efficiency of our approach by localizing a mobile quadcopter using custom ultra-wideband wireless devices. Our code and data are available online.