Lagrangian data assimilation is a complex problem in oceanic and atmospheric modeling. Tracking drifters in large-scale geophysical flows can involve uncertainty in drifter location, complex inertial effects, and other factors which make comparing them to simulated Lagrangian trajectories from numerical models extremely challenging. Temporal and spatial discretization, factors necessary in modeling large scale flows, also contribute to separation between real and simulated drifter trajectories. The chaotic advection inherent in these turbulent flows tends to separate even closely spaced tracer particles, making error metrics based solely on drifter displacements unsuitable for estimating model parameters. We propose to instead use error in the coherent structure coloring (CSC) field to assess model skill. The CSC field provides a spatial representation of the underlying coherent patterns in the flow, and we show that it is a more robust metric for assessing model accuracy. Through the use of two test cases, one considering spatial uncertainty in particle initialization, and one examining the influence of stochastic error along a trajectory and temporal discretization, we show that error in the coherent structure coloring field can be used to accurately determine single or multiple simultaneously unknown model parameters, whereas a conventional error metric based on error in drifter displacement fails. Because the CSC field enhances the difference in error between correct and incorrect model parameters, error minima in model parameter sweeps become more distinct. The effectiveness and robustness of this method for single and multi-parameter estimation in analytical flows suggests that Lagrangian data assimilation for real oceanic and atmospheric models would benefit from a similar approach.