Automatic speaker verification (ASV) with ad-hoc microphone arrays has received attention. Unlike traditional microphone arrays, the number of microphones and their spatial arrangement in an ad-hoc microphone array is unknown, which makes conventional multi-channel ASV techniques ineffective in ad-hoc microphone array settings. Recently, an utterance-level ASV with ad-hoc microphone arrays has been proposed, which first extracts utterance-level speaker embeddings from each channel of an ad-hoc microphone array, and then fuses the embeddings for the final verification. However, this method cannot make full use of the cross-channel information. In this paper, we present a novel multi-channel ASV model at the frame-level. Specifically, we add spatio-temporal processing blocks (STB) before the pooling layer, which models the contextual relationship within and between channels and across time, respectively. The channel-attended outputs from STB are sent to the pooling layer to obtain an utterance-level speaker representation. Experimental results demonstrate the effectiveness of the proposed method.