Glioma is one of the most common types of brain tumors arising in the glial cells in the human brain and spinal cord. In addition to the threat of death, glioma treatment is also very costly. Hence, automatic and accurate segmentation and measurement from the early stages are critical in order to prolong the survival rates of the patients and to reduce the costs of health care. In the present work, we propose a novel end-to-end cascaded network for semantic segmentation that utilizes the hierarchical structure of the tumor sub-regions with ResNet-like blocks and Squeeze-and-Excitation modules after each convolution and concatenation block. By utilizing cross-validation, an average ensemble technique, and a simple post-processing technique, we obtained dice scores of 90.34, 81.12, and 78.42 and Hausdorff Distances (95th percentile) of 4.32, 6.28, and 3.70 for the whole tumor, tumor core, and enhancing tumor, respectively, on the online validation set.