EU-Agent-Bench: Measuring Illegal Behavior of LLM Agents Under EU Law

Large language models (LLMs) are increasingly deployed as agents in various contexts by providing tools at their disposal. However, LLM agents can exhibit unpredictable behaviors, including taking undesirable and/or unsafe actions. In order to measure the latent propensity of LLM agents for taking illegal actions under an EU legislative context, we introduce EU-Agent-Bench, a verifiable human-curated benchmark that evaluates an agent's alignment with EU legal norms in situations where benign user inputs could lead to unlawful actions. Our benchmark spans scenarios across several categories, including data protection, bias/discrimination, and scientific integrity, with each user request allowing for both compliant and non-compliant execution of the requested actions. Comparing the model's function calls against a rubric exhaustively supported by citations of the relevant legislature, we evaluate the legal compliance of frontier LLMs, and furthermore investigate the compliance effect of providing the relevant legislative excerpts in the agent's system prompt along with explicit instructions to comply. We release a public preview set for the research community, while holding out a private test set to prevent data contamination in evaluating upcoming models. We encourage future work extending agentic safety benchmarks to different legal jurisdictions and to multi-turn and multilingual interactions. We release our code on \href{https://github.com/ilijalichkovski/eu-agent-bench}{this URL}.

Feature Characterization for Profile Surface Texture

Conventional field parameters for surface measurement use all data points, while feature characterization focuses on subsets extracted by watershed segmentation. This approach enables the extraction of specific features that are potentially responsible for the function of the surface or are a direct reflection of the manufacturing process, allowing for a more accurate assessment of both aspects. Feature characterization with the underlying watershed segmentation for areal surface topographies has been standardized for over a decade and is well established in industry and research. In contrast, feature characterization for surface profiles has been standardized recently, and the corresponding standard for watershed segmentation is planned to be published in the near future. Since the standards do not provide guidelines for implementation, this paper presents an unambiguous algorithm of the watershed segmentation and the feature characterization for surface profiles. This framework provides the basis for future work, mainly investigating the relationship between feature parameters based on feature characterization and the function of the surface or manufacturing process. For this purpose, recommendations for the configuration and extensions of the toolbox can also be developed, which could find their way into the ISO standards.