Unmanned aerial vehicle (UAV) communication is of crucial importance for diverse practical applications. However, it is susceptible to the severe spectrum scarcity problem and interference since it operates in the unlicensed spectrum band. In order to tackle those issues, a dynamic spectrum sharing network is considered with the anti-jamming technique. Moreover, an intelligent spectrum allocation and trajectory optimization scheme is proposed to adapt to diverse jamming models by exploiting our designed novel online-offline multi-agent actor-critic and deep deterministic policy-gradient framework. Simulation results demonstrate the high efficiency of our proposed framework. It is also shown that our proposed scheme achieves the largest transmission rate among all benchmark schemes.