We present an online algorithm for reconstructing a signal from a set of non-uniform samples. By representing the signal using compactly supported basis functions, we show how estimating the expansion coefficients using least-squares can be implemented in a streaming manner: as batches of samples over subsequent time intervals are presented, the algorithm forms an initial estimate of the signal over the sampling interval then updates its estimates over previous intervals. We give conditions under which this reconstruction procedure is stable and show that the least-squares estimates in each interval converge exponentially, meaning that the updates can be performed with finite memory with almost no loss in accuracy. We also discuss how our framework extends to more general types of measurements including time-varying convolution with a compactly supported kernel.