This contribution reviews critically the existing entropy measures for probabilistic hesitant fuzzy sets (PHFSs), and demonstrates that these entropy measures fail to effectively distinguish a variety of different PHFSs in some cases. In the sequel, we develop a new axiomatic framework of entropy measures for probabilistic hesitant fuzzy elements (PHFEs) by considering two facets of uncertainty associated with PHFEs which are known as fuzziness and nonspecificity. Respect to each kind of uncertainty, a number of formulae are derived to permit flexible selection of PHFE entropy measures. Moreover, based on the proposed PHFE entropy measures, we introduce some entropy-based distance measures which are used in the portion of comparative analysis.