Ultrasound localization microscopy (ULM) refers to a promising medical imaging modality that systematically leverages the advantages of contrast-enhanced ultrasound (CEUS) to surpass the diffraction barrier and delineate the microvascular map. Localization and tracking of microbubbles (MBs), two significant steps of ULM, facilitate generating the vascular map and the velocity distribution, respectively. Herein, we propose a novel MB tracking technique considering temporal pairing as a bubble-set registration problem. Iterative registration is performed between the bubble sets in two consecutive time instants by analytically optimizing a cost function that takes position and point-spread function (PSF) similarities as well as physically plausible levels of bubbles' movement into account. Furthermore, we infer MBs' parity in a fuzzy manner instead of binary. The proposed technique performs well in validation experiments with two synthetic and two in vivo datasets provided by the Ultrasound Localisation and TRacking Algorithms for Super Resolution (ULTRA-SR) Challenge.