This paper proposes an ultrasonic backscatter communication (UsBC) system for passive implantable medical devices (IMDs) that can operate without batteries, enabling versatile revolutionary applications for future healthcare. The proposed UsBC system consists of a reader and a tag. The reader sends interrogation pulses to the tag. The tag backscatters the pulses based on the piezoelectric effect of a piezo transducer. We present several basic modulation schemes for UsBC by impedance matching of the piezo transducer. To mitigate the interference of other scatters in the human body, the tag transmits information bits by codeword mapping, and the reader performs codeword matching before energy detection in the reader. We further derive the theoretical bit-error rate (BER) expression. Monte Carlo simulations verify the theoretical analysis and show that passive UsBC can achieve low BER and low complexity, which is desirable for size- and energy-constrained IMDs.