Over the last decade, several studies have investigated the weaknesses of Android malware detectors against adversarial examples by proposing novel evasion attacks; however, the practicality of most studies in manipulating real-world malware is arguable. The majority of studies have assumed attackers know the details of the target classifiers used for malware detection, while in real life, malicious actors have limited access to the target classifiers. This paper presents a practical evasion attack, EvadeDroid, to circumvent black-box Android malware detectors. In addition to generating real-world adversarial malware, the proposed evasion attack can preserve the functionality of the original malware samples. EvadeDroid applies a set of functionality-preserving transformations to morph malware instances into benign ones using an iterative and incremental manipulation strategy. The proposed manipulation technique is a novel, query-efficient optimization algorithm with the aim of finding and injecting optimal sequences of transformations into malware samples. Our empirical evaluation demonstrates the efficacy of EvadeDroid under hard- and soft-label attacks. Moreover, EvadeDroid is capable to generate practical adversarial examples with only a small number of queries, with evasion rate of 81%, 73%, and 75% for DREBIN, Sec-SVM, and MaMaDroid, respectively. Finally, we show that EvadeDroid is able to preserve its stealthiness against four popular commercial antivirus, thus demonstrating its feasibility in the real world.