Knowledge Graph Embeddings (KGEs) have shown promising performance on link prediction tasks by mapping the entities and relations from a knowledge graph into a geometric space (usually a vector space). Ultimately, the plausibility of the predicted links is measured by using a scoring function over the learned embeddings (vectors). Therefore, the capability in preserving graph characteristics including structural aspects and semantics highly depends on the design of the KGE, as well as the inherited abilities from the underlying geometry. Many KGEs use the flat geometry which renders them incapable of preserving complex structures and consequently causes wrong inferences by the models. To address this problem, we propose a neuro differential KGE that embeds nodes of a KG on the trajectories of Ordinary Differential Equations (ODEs). To this end, we represent each relation (edge) in a KG as a vector field on a smooth Riemannian manifold. We specifically parameterize ODEs by a neural network to represent various complex shape manifolds and more importantly complex shape vector fields on the manifold. Therefore, the underlying embedding space is capable of getting various geometric forms to encode complexity in subgraph structures with different motifs. Experiments on synthetic and benchmark dataset as well as social network KGs justify the ODE trajectories as a means to structure preservation and consequently avoiding wrong inferences over state-of-the-art KGE models.