Dialogue systems in the open domain have achieved great success due to large conversation data and the development of deep learning, but multi-turn systems are often restricted with the frequent coreference and information omission. In this paper, we investigate the incomplete utterance restoration since it has brought general improvement over multi-turn dialogue systems in different domains. In the task, we propose a novel semi autoregressive generator (SARG) with the high efficiency and flexibility, which is inspired by the autoregression for generation and the sequence labeling for overlapped rewriting. Moreover, experiments on \textit{Restoration-200k} show that our proposed model significantly outperforms the state-of-the-art models with faster inference speed.