This paper proposes a word representation strategy for rhythm patterns. Using 1034 pieces of Nottingham Dataset, a rhythm word dictionary whose size is 450 (without control tokens) is generated. BERT model is created to explore syntactic potentials of rhythm words. Our model is able to find overall music structures and cluster different meters. In a larger scheme, a think mode - music as language - is proposed for systematic considerations.