Since the last two decades, luggage scanning has become one of the prime aviation security concerns all over the world. Manual screening of the baggage items is a cumbersome, subjective and inefficient process and many researchers have developed x-ray imagery based autonomous systems for this purpose. However, to the best of our knowledge, there is no framework till date which can recognize heavily occluded and cluttered baggage items from x-ray scans irrespective of the acquisition machinery or the scan modality. This paper presents a deep cascaded multiscale structure tensor-based framework which can automatically extract and recognize normal as well as suspicious items irrespective of their position and orientation from the multivendor x-ray scans. The proposed framework is unique in its kind as it intelligently extracts each object by iteratively picking contour based transitional information from different orientations and uses only a single feedforward convolutional neural network for the recognition. The proposed framework has been rigorously tested on two publicly available datasets containing cumulative of 1,067,381 x-ray scans where it significantly outperformed the existing state-of-the-art solutions by achieving the mean intersection-over-union ratings of up to 0.9689, area under the curve of up to 0.9950, accuracy of up to 0.9955 and the mean average precision score of up to 0.9453 for detecting normal as well as suspicious baggage items. Furthermore, the proposed framework has achieved 15.78% better time performance as compared to the popular object detectors.