In this paper, a neural-network (NN)-based online optimal control method (NN-OPT) is proposed for ultra-capacitors (UCs) energy storage system (ESS) in hybrid AC/DC microgrids involving multiple distributed generations (e.g., Photovoltaic (PV) system, battery storage, diesel generator). Conventional control strategies usually produce large disturbances to buses during charging and discharging (C&D) processes of UCs, which significantly degrades the power quality and system performance, especially under fast C&D modes. Therefore, the optimal control theory is adopted to optimize the C&D profile as well as to suppress the disturbances caused by UCs implementation. Specifically, an NN-based intelligent algorithm is developed to learn the optimal control policy for bidirectional-converter-interfaced UCs. The inaccuracies of system modeling are also considered in the control design. Since the designed NN-OPT method is decentralized that only requires the local measurements, plug & play of UCs can be easily realized with minimal communication efforts. In addition, the PV system is under the maximum power point tracking (MPPT) control to extract the maximum benefit. Both islanded and grid-tied modes are considered during the controller design. Extensive case studies have been conducted to evaluate the effectiveness of the proposed method.