Delayed bit-interleaved coded modulation (DBICM) generalizes bit-interleaved coded modulation (BICM) by modulating differently delayed sub-blocks of codewords onto the same signals. DBICM improves transmission reliability over BICM due to its capability of detecting undelayed sub-blocks with the extrinsic information of the decoded delayed sub-blocks. In this work, we propose a novel windowed decoding algorithm for DBICM, which uses the extrinsic information of both the decoded delayed and undelayed sub-blocks, to improve the detection on all sub-blocks. Numerical results show that the proposed windowed decoding significantly outperforms the original decoding.