We introduce a framework that unifies the existing work on black-box adversarial example generation. We demonstrate that the current state of the art in the field is optimal in a certain natural sense. Despite this optimality, we show how to improve black-box attacks by bringing a new element into the problem: ambient priors for the gradient. We identify two such priors, and give an algorithm based on bandit optimization that allows for seamless integration of these and other priors. Our framework leads to methods that are two to three times more query-efficient and two to three times smaller failure rate than the state-of-the-art approaches.