Echo State Networks (ESN) are a class of Recurrent Neural Networks (RNN) that has gained substantial popularity due to their effectiveness, ease of use and potential for compact hardware implementation. An ESN contains the three network layers input, reservoir and readout where the reservoir is the truly recurrent network. The input and reservoir layers of an ESN are initialized at random and never trained afterwards and the training of the ESN is applied to the readout layer only. The alternative of Recursive Neural Gas (RNG) is one of the many proposals of fully-trainable reservoirs that can be found in the literature. Although some improvements in performance have been reported with RNG, to the best of authors' knowledge, no experimental comparative results are known with benchmarks for which ESN is known to yield excellent results. This work describes an accurate model of RNG together with some extensions to the models presented in the literature and shows comparative results on three well-known and accepted datasets. The experimental results obtained show that, under specific circumstances, RNG-based reservoirs can achieve better performance.