LLM-Derived Preference Judgments Are Not Self-Consistent

Agents increasingly interpret a person's natural-language preferences by querying an LLM for numerical preference judgments, e.g., by asking how much the person would be willing to pay for an item. A growing body of work estimates a utility function from these judgments and then chooses actions based on their estimated utility. This pipeline assumes the judgments are approximately self-consistent: that a single utility function can reproduce them. But are they? To study this question, we measure the self-consistency of cardinal LLM preference judgments. For example, the difference in stated willingness-to-pay between two items should match the stated payment that makes a person indifferent to exchanging them. We develop statistical tests and interpretable measures of how far observed responses depart from the best-fitting self-consistent utility function. Experiments with flight, apartment, and hotel examples across six LLMs reveal large persistent inconsistencies. This suggests that LLM-derived preference judgments cannot be faithfully summarized by a single utility function.

An Interpretable Determinantal Choice Model for Subset Selection

Understanding how subsets of items are chosen from offered sets is critical to assortment planning, wireless network planning, and many other applications. There are two seemingly unrelated subset choice models that capture dependencies between items: intuitive and interpretable random utility models; and tractable determinantal point processes (DPPs). This paper connects the two. First, all DPPs are shown to be random utility models. Next, a determinantal choice model that enjoys the best of both worlds is specified; the model is shown to subsume logistic regression when dependence is minimal, and MNL when dependence is maximally negative. This makes the model interpretable, while retaining the tractability of DPPs. A simulation study verifies that the model can learn a continuum of negative dependencies from data, and an applied study using original experimental data produces novel insights on wireless interference in LoRa networks.