This research studies graph-based approaches for Answer Sentence Selection (AS2), an essential component for building retrieval-based Question Answering systems. Given a question, our model creates a small-scale relevant training graph to perform more accurate AS2. The nodes of the graphs are question-answer pairs, where the answers are also sentences. We train and apply state-of-the-art models for computing scores between question-question, question-answer, and answer-answer pairs. We apply thresholding to the relevance scores for creating edges between nodes. Finally, we apply Graph Neural Networks to the obtained graph to perform joint learning and inference for solving the AS2 task. The experiments on two well-known academic benchmarks and a real-world dataset show that our approach consistently outperforms state-of-the-art models.