This letter proposes a neural network (NN) aided remote unmanned aerial vehicle (UAV) online control algorithm, coined oHJB. By downloading a UAV's state, a base station (BS) trains an HJB NN that solves the Hamilton-Jacobi-Bellman equation (HJB) in real time, yielding the optimal control action. Initially, the BS uploads this control action to the UAV. If the HJB NN is sufficiently trained and the UAV is far away, the BS uploads the HJB NN model, enabling to locally carry out control decisions even when the connection is lost. Simulations corroborate the effectiveness of oHJB in reducing the UAV's travel time and energy by utilizing the trade-off between uploading delays and control robustness in poor channel conditions.