Due to the naturally power-law distributed nature of user-item interaction data in recommendation tasks, hyperbolic space modeling has recently been introduced into collaborative filtering methods. Among them, hyperbolic GCN combines the advantages of GCN and hyperbolic space and achieves a surprising performance. However, these methods only partially exploit the nature of hyperbolic space in their designs due to completely random embedding initialization and an inaccurate tangent space aggregation. In addition, the data used in these works mainly focus on user-item interaction data only, which further limits the performance of the models. In this paper, we propose a hyperbolic GCN collaborative filtering model, HGCC, which improves the existing hyperbolic GCN structure for collaborative filtering and incorporates side information. It keeps the long-tailed nature of the collaborative graph by adding power law prior to node embedding initialization; then, it aggregates neighbors directly in multiple hyperbolic spaces through the gyromidpoint method to obtain more accurate computation results; finally, the gate fusion with prior is used to fuse multiple embeddings of one node from different hyperbolic space automatically. Experimental results on four real datasets show that our model is highly competitive and outperforms leading baselines, including hyperbolic GCNs. Further experiments validate the efficacy of our proposed approach and give a further explanation by the learned embedding.