Predicting the potential success of a book in advance is vital in many applications. This could help both publishers and readers in their decision making process whether or not a book is worth publishing and reading, respectively. This prediction could also help authors decide whether a book draft is good enough to send to a publisher. We propose a model that leverages Convolutional Neural Networks along with readability indices. Unlike previous methods, our method includes no count-based, lexical, or syntactic hand-crafted features. Instead, we make use of a pre-trained sentence encoder to encode the book sentences. We highlight the connection between this task and book genre identification by showing that embeddings that are good at capturing the separability of book genres are better for the book success prediction task. We also show that only the first 1K sentences are good enough to predict the successability of books. Our proposed model outperforms strong baselines on this task by as large as 6.4% F1-score.