Beyond Static Evaluation: Building Simulation Environments for Scalable Agentic Reinforcement Learning

As Large Language Models (LLMs) evolve into autonomous agents, traditional static evaluation fails to capture multi-step decision-making. We introduce AgenticAI-Supervisor, an API and UI-driven RL Gym environment that decouples environment creation from scalable execution. By moving to verifiable execution outcomes, the platform generates high-fidelity traces and applies multi-dimensional reward shaping. Critically, our framework mitigates reward hacking through rigorous internal state validation and testing. This work provides a first look at our platform's core capabilities through a Customer Support Agent case study demonstrating a consistent closed-loop feedback for model optimization. Future work will focus on advanced features such as Computer Use, Tool Use, automated "stumping", and edge-case generation.

Improving feature interactions at Pinterest under industry constraints

Adopting advances in recommendation systems is often challenging in industrial settings due to unique constraints. This paper aims to highlight these constraints through the lens of feature interactions. Feature interactions are critical for accurately predicting user behavior in recommendation systems and online advertising. Despite numerous novel techniques showing superior performance on benchmark datasets like Criteo, their direct application in industrial settings is hindered by constraints such as model latency, GPU memory limitations and model reproducibility. In this paper, we share our learnings from improving feature interactions in Pinterest's Homefeed ranking model under such constraints. We provide details about the specific challenges encountered, the strategies employed to address them, and the trade-offs made to balance performance with practical limitations. Additionally, we present a set of learning experiments that help guide the feature interaction architecture selection. We believe these insights will be useful for engineers who are interested in improving their model through better feature interaction learning.

NxtPost: User to Post Recommendations in Facebook Groups

In this paper, we present NxtPost, a deployed user-to-post content-based sequential recommender system for Facebook Groups. Inspired by recent advances in NLP, we have adapted a Transformer-based model to the domain of sequential recommendation. We explore causal masked multi-head attention that optimizes both short and long-term user interests. From a user's past activities validated by defined safety process, NxtPost seeks to learn a representation for the user's dynamic content preference and to predict the next post user may be interested in. In contrast to previous Transformer-based methods, we do not assume that the recommendable posts have a fixed corpus. Accordingly, we use an external item/token embedding to extend a sequence-based approach to a large vocabulary. We achieve 49% abs. improvement in offline evaluation. As a result of NxtPost deployment, 0.6% more users are meeting new people, engaging with the community, sharing knowledge and getting support. The paper shares our experience in developing a personalized sequential recommender system, lessons deploying the model for cold start users, how to deal with freshness, and tuning strategies to reach higher efficiency in online A/B experiments.