COVID-19 is a highly contagious respiratory infection that has affected a large population across the world and continues with its devastating consequences. It is imperative to detect COVID-19 at the earliest to limit the span of infection. In this work, a new classification technique CB-STM-RENet based on deep Convolutional Neural Network (CNN) and Channel Boosting is developed for the detection of COVID-19 from chest X-Rays. In this connection, to learn the COVID-19 specific radiographic patterns, a new convolution block based on split-transform-merge (STM) is developed. This new block systematically incorporates region and edge base operations at each branch to capture the diverse set of features at various levels, especially those related to region homogeneity, textural variations, and boundaries of the infected region. We further enhanced the learning and discrimination capability of the proposed CNN architecture by exploiting the Channel Boosting idea that concatenates the auxiliary channels along with the original channels. The auxiliary channels are generated by employing Transfer Learning-based domain adaption of the pre-trained CNN architectures. The effectiveness of the proposed technique CB-STM-RENet is evaluated on three different datasets containing CoV-Healthy-6k, CoV-NonCoV-10k, and CoV-NonCoV-15k Chest X-Ray images, respectively. The performance comparison of the proposed deep CB-STM-RENet with the existing techniques exhibits high classification performance both in discriminating COVID-19 chest infections from healthy, as well as, other types of chest infections. CB-STM-RENet provides the highest performance on all these three datasets; especially on the stringent CoV-NonCoV-15k dataset. The good detection rate (97%), and high precision (93%) of the proposed technique suggest that it can be adapted for the diagnosis of COVID-19 infected patients.