We present a decentralized trajectory optimization scheme based on learning model predictive control for multi-agent systems with nonlinear decoupled dynamics under separable cost and coupled state constraints. By performing the same task iteratively, data from previous task executions is used to construct and improve local time-varying safe sets and an approximate value function. These are used in a decentralized MPC problem as the terminal sets and terminal cost function. Our framework results in a decentralized controller, which requires no communication between agents over each iteration of task execution, and guarantees persistent feasibility, closed-loop stability, and non-decreasing performance of the global system over task iterations. Numerical experiments of a multi-vehicle collision avoidance scenario demonstrate the effectiveness of the proposed scheme.