In games (as well as many user-facing systems), adapting content to user's preferences and experience is an important challenge. This paper explores a novel method to realize this goal in the context of dynamic difficulty adjustment (DDA). Here the aim is to constantly adapt the content of a game to the skill level of the player, keeping them engaged by avoiding states that are either too difficult or too easy. Current systems for DDA rely on expensive data mining, or on hand-crafted rules designed for particular domains, and usually adapts to keep players in the flow, leaving no room for the designer to present content that is purposefully easy or difficult. This paper presents a Bayesian Optimization-based system for DDA that is agnostic to the domain and that can target particular difficulties. We deploy this framework in two different domains: the puzzle game Sudoku, and a simple Roguelike game. By modifying the acquisition function's optimization, we are reliably able to present a puzzle with a bespoke difficulty for players with different skill levels in less than five iterations (for Sudoku) and fifteen iterations (for the simple Roguelike), significantly outperforming simpler heuristics for difficulty adjustment in said domains, with the added benefit of maintaining a model of the user. These results point towards a promising alternative for content adaption in a variety of different domains.