The technical skill of surgeons directly impacts patient outcomes. Advanced tracking systems enable the development of objective motion-based metrics for skill evaluation, but these metrics are not sufficient to evaluate the performance in complex surgical tasks. In this study, we developed metrics for surgical skill evaluation that are based on the orientation of the surgical instruments. Experienced robotic surgeons and novice users performed teleoperated (using the da Vinci Research Kit) and open needle-driving. Task time and the rate of orientation change successfully distinguished between experienced surgeons and novice users. Path length and the normalized angular displacement allowed for a good separation only in part of the experiment. Our new promising metrics for surgical skill evaluation captured technical aspects that are taught during surgeons' training. They provide complementing evaluation to those of classical metrics. Orientation-based metrics add value to skill assessment and may be an adjunct to classic objective metrics providing more granular discrimination of skills.