Tuberculosis is an infectious disease that is leading to the death of millions of people across the world. The mortality rate of this disease is high in patients suffering from immuno-compromised disorders. The early diagnosis of this disease can save lives and can avoid further complications. But the diagnosis of TB is a very complex task. The standard diagnostic tests still rely on traditional procedures developed in the last century. These procedures are slow and expensive. So this paper presents an automatic approach for the diagnosis of TB from posteroanterior chest x-rays. This is a two-step approach, where in the first step the lung regions are segmented from the chest x-rays using the graph cut method, and then in the second step the transfer learning of VGG16 combined with Bi-directional LSTM is used for extracting high-level discriminative features from the segmented lung regions and then classification is performed using a fully connected layer. The proposed model is evaluated using data from two publicly available databases namely Montgomery Country set and Schezien set. The proposed model achieved accuracy and sensitivity of 97.76%, 97.01% and 96.42%, 94.11% on Schezien and Montgomery county datasets. This model enhanced the diagnostic accuracy of TB by 0.7% and 11.68% on Schezien and Montgomery county datasets.