In this paper, we propose an end-to-end learning network aim at predicting future PC frames, based on point-based RNN network. As main novelty, an initial layer learns topological information of point clouds as geometric features and then uses the learned features to form representative spatio-temporal neighborhoods. This module is followed by multiple Graph-RNN cells. Each cell learns points dynamics (i.e., RNN states) processing each point jointly with the spatio-temporal neighboring points. We tested the network performance with a MINST dataset of moving digits, a synthetic human bodies motions and JPEG dynamic bodies datasets. Simulation results demonstrated that our method outperforms baseline ones that neglect geometry