Hundreds of thousands of mobile app users post their reviews online including feedback and queries. Responding to user reviews promptly and satisfactorily improves application ratings, which is key to application popularity and success. The proliferation of such reviews makes it virtually impossible for developers to keep up with responding manually. To address the challenge, recent work has shown the possibility of automatic response generation by training a seq2seq model with a large collection of review-response pairs. However, because the training review-response pairs are aggregated from many different apps, it remains challenging to let the model generate app-specific responses, which, on the other hand, are often desirable as apps have different features and concerns. To enable app-specific response generation, this work proposes AARSynth: an app-aware response synthesis system. The key idea behind AARSynth is that it augments the seq2seq model with information specific to the app. Given an input review, AARSynth first retrieves the top-K most relevant app reviews and the most relevant snippet from the app description. The retrieved information, together with the input review, is then fed into a fused machine learning model that integrates the seq2seq model with a machine reading comprehension model. The latter helps digest the relevant reviews and app description. Finally, the fused model generates a response that is customized to the given app. We evaluated AARSynth using a large corpus of reviews and responses from Google Play. The results show that AARSynth outperforms the state-of-the-art system by 22.20% on the BLEU-4 score. Furthermore, our human study shows that AARSynth produces a statistically significant improvement in response quality compared to the state-of-the-art system.