Abstract:We reconstruct the mentor--student network through which documented scholarly training passed across roughly nine centuries, and subject both the network and the means of reconstructing it to source criticism. From Wikidata, which aggregates the Mathematics Genealogy Project and the MacTutor Archive, we extract approximately 470,000 mentor--student assertions, yielding a directed acyclic graph of 372,853 persons. Using all 64 historical Fields Medalists as a fixed, ex ante tracer set, backward traversal enumerates some 25.5 million distinct paths reaching 57 generations. Three structural observations follow. Genealogical traffic through Leibniz forms an hourglass: thin upstream, 5.3 paths per node on average, and thick downstream, 53.4, a ratio near 10:1, with no counterpart at Newton, who lies on only four of the 64 lineages. Across a window centered on Leibniz, seven independently extracted predicate dimensions reorganize together, and recorded learned-society membership rises from 6.5 to 82.1 percent of the cohort. Upstream, 54 of the 64 lineages converge on the same five twelfth- and thirteenth-century Islamic and Byzantine scholars before terminating at an eleventh-century boundary we name the Monastery Wall. We argue that such observations cannot be assessed without tool criticism. The traversal engine is algebraically reversible, so every ranking decision it makes can be reconstructed afterward. We characterize its measurement bias in closed form, show that the macro-structures survive switching that bias off, and report the family of lineages the traversal returns at different resolutions rather than a single ranked list.
Abstract:Despite the remarkable recent progress of video world models, social interaction between users and the characters within these worlds remains unsupported. To fill this gap, we present HelloWorld, a video world model that enables social interaction with in-world characters. With a single button press, users can prompt the on-screen character to respond toward the camera, e.g., turning to the viewer, waving, nodding, or speaking a short greeting. To make these interactions natural, we propose a self-distillation pipeline that finetunes the video generation model on data synthesized by itself. Each synthesized clip contains both social interactions and camera motion, allowing the model to learn camera-pose conditioning without degrading interaction quality. At inference, we further introduce a training-free module that determines when the interaction occurs. Upon a button press, it modulates the cross-attention masks of the DiT so that the interaction-related text prompt attends only to the frames within the press window, temporally localizing the character's response. We further build HelloWorldBench, a 400-sample benchmark with three social interaction metrics alongside three conventional metrics, for evaluation. Experiments demonstrate that HelloWorld surpasses a variety of baselines in interaction quality, while maintaining state-of-the-art picture aesthetics and camera-pose following. Project page: https://github.com/AlayaLab/HelloWorld
Abstract:Recent years have witnessed an explosive trend of scaling ego-centric human videos for robot manipulation, yet it remains unclear which data actually benefits dexterous manipulation. We present SiMDex, a similarity-based data mining framework that casts human data selection for VLA post-training in dexterous manipulation as a recommendation problem. For each robot demonstration, SiMDex employs a three-layer recall-ranking-re-ranking pipeline to extract task-relevant subsets from a pool of ~32M egocentric human samples, operating in a morphology-agnostic action space that requires no changes to VLA architecture or training. Against a strong baseline trained with an equal amount of randomly sampled human data, SiMDex uses only ~1.49M mined samples (<5% of the pool) yet improves the overall success rate from 47.7% to 61.1%, showing that selective curation outperforms indiscriminate data mixing.
Abstract:In The Algebraic Mind, Marcus identified three cognitive components: operations over variables, recursively structured representations, and an individual/kind distinction. He left the neural substrate open. A companion paper solves this with VaCoAl, an architecture built on GF(2) XOR-and-shift. It offers exact reversible variable binding, compositional bundling, and spatial individual/kind separation. A collision-avoidance scheme ensures exact recovery within a Frontier Size. Its PyVaCoAl implementation demonstrates this algebra scales to millions of records, while the SRAM-CAM version favors hardware speed. Marcus's horizontal framework concerns how a mind represents the world. This paper reads VaCoAl through an orthogonal lens: Minsky's The Emotion Machine, providing the vertical dimension of a mind reasoning about itself. We claim VaCoAl's exact preservation and recovery of structural representations directly enables introspection -- Minsky's Reflective layer. Reflexively unbinding one's own deliberative trace is the operation. We grade deliberation into three tiers by trace faithfulness: probabilistic (LLMs), approximate-algebraic, and exact-algebraic (VaCoAl). Only exact reversibility ensures reflexive recovery is faithful rather than spurious, reaching the Reflective layer. This substrate carries two more Minsky ideas: panalogy (achieved via content-addressable retrieval where match accuracy indicates analogical fit, enabling lifelong learning) and credit assignment (framed as reflexive counterfactual simulation: surgically unbinding a past choice and rebinding a counterfactual to compare). The exact reversibility allowing the companion paper to climb Pearl's causal ladder helps this paper ascend Minsky's introspective stack, finding the intersection. We separate demonstrated trace execution from theoretical Reflection arguments, leaving meta-control loops strictly for future work.
Abstract:When should an intelligent assistant speak up without being asked? Continuous egocentric video offers rich, evolving context that enables a new form of assistance: one that is proactive rather than merely reactive. Yet existing approaches either wait passively for user queries or treat every detected event as requiring a response, without considering the user's history, current activity, or whether assistance would actually be welcome. We reframe proactive assistance as a context-dependent decision problem: the agent must not only perceive what is happening, but reason over accumulated temporal context to determine when and whether to intervene. To this end, we present Vinci2, a proactive egocentric assistance system that advances the on-device assistant Vinci from reactive response toward proactivity. On the evaluation side, we present EgoServe, the first large-scale benchmark for proactive assistance in continuous egocentric video. EgoServe comprises over 3,000 service instances organized along 4 temporal memory horizons, ranging from immediate safety alerts to long-term habit coaching, across 10 service categories. On the modeling side, we propose EgoMemo, a training-free, memory-augmented agent that maintains three complementary memory representations: multi-scale temporal summaries, a semantic knowledge graph, and visual embedding archives. At each timestep, EgoMemo performs retrieval-augmented reasoning to determine whether assistance is warranted and, if so, produces contextually grounded responses. Experiments demonstrate that EgoMemo establishes strong baselines on EgoServe while remaining competitive on existing egocentric benchmarks. Our benchmark and code are publicly available at \href{https://sitonggong.github.io/EgoServe-page/}{Vinci2}.
Abstract:Hand-object interaction (HOI) recognition requires capturing both hand manipulations and object transformations. However, existing video-language models often fall into shortcuts by relying on spurious correlations among hands, objects, or environmental context, rather than reasoning from the appearance and dynamics of hands and objects themselves. To address this limitation, we propose a new learning paradigm that combines (i) hand-object masked training, which enables robust reasoning from partial hand or object observations, and (ii) an HOI-dynamics-aware decoder that explicitly learns hand- and object-centric embeddings through auxiliary predictions of their locations and semantics, enhancing sensitivity to both cues. To systematically evaluate such cue-specific reasoning, we introduce Cue-Isolated HOI (CI-HOI), a new evaluation that assesses models' ability to predict actions from hand- and object-related cues independently. To enable CI-HOI, we curate the DEHOI testbed, which separates hand- and object-related observations for disentangled HOI evaluation through inpainting. Using DEHOI, we demonstrate both quantitatively and qualitatively that our training strategy exploits hand- and object-centric information more effectively than existing models. Our approach improves over existing models on DEHOI, standard action recognition, object state recognition, and even robot manipulation action recognition, leading to more robust HOI understanding.
Abstract:What made useful knowledge cumulative was not discovery alone but the institutions that transmitted it. We provide the first exhaustive structural measurement of the network through which upper-tail human capital passed from master to student across a millennium. Using 470,000 mentor-student records from Wikidata (which integrates the Mathematics Genealogy Project and MacTutor Archive), and all 64 historical Fields Medalists as a fixed, ex ante tracer set, backward traversal yields a directed acyclic graph of 25.5 million paths reaching 57 generations. We document two institutional transitions. First, a 17th-century watershed concentrates lineage traffic on Leibniz: 47 of 64 lineages pass through him with a 10:1 downstream-to-upstream ratio, and seven independent attributes -- learned-society membership (a 46-fold rise per scholar), field, language, employer, institutional diversification, student production, and diffusion entropy -- re-organize coherently across the same window. This is the network signature of Mokyr's Republic of Letters, and it reframes the Newton-Leibniz priority dispute as a distinction between the possession and the transmission of upper-tail human capital: it is transmission that generates the spillovers on which growth depends. Second, 84% of lineages converge upstream on five 12th-13th-century Islamic and Byzantine scholars before terminating at an 11th-century boundary -- the ``Monastery Wall'' -- at which personal academic mentorship first becomes record-generating in Europe. Our claims are descriptive-structural, not causal. Because exhaustive traversal at this scale defeats standard tools, we also contribute a deterministic, algebraic graph-traversal instrument whose measurement bias we characterize in closed form, and report one emergent property of independent methodological interest.
Abstract:Cause-and-effect reasoning in video is a significant challenge for Vision-Language Models (VLMs), as it requires going beyond surface-level perception to a deeper understanding of causal mechanisms. However, existing benchmarks rarely provide the fine-grained, grounded evidence needed to rigorously evaluate this capability. To address this gap, we introduce CaST-Bench, a benchmark for Causal Chain-Grounded Spatio-Temporal Video Reasoning. CaST-Bench presents complex causal questions that require models to identify and localize a chain of multiple spatio-temporal evidences. Through a human-AI collaborative pipeline, we construct a high-quality dataset of 2,066 questions over 1,015 videos, with causal chains annotated by temporal segments and bounding-box tracks. Furthermore, we design a comprehensive evaluation suite with novel metrics that assess not only answer correctness but also the capability for visual evidence grounded reasoning. This grounding is crucial for improving accuracy by mitigating spurious correlations and for enhancing user trust by making models more transparent. Our experiments show that current VLMs struggle with causal questions, largely due to their limited ability to construct precise and grounded causal chains. This highlights an important direction for improving future VLMs.
Abstract:Multimodal Large Language Models (MLLMs) are increasingly deployed in human-facing roles where personality perception is critical, yet existing benchmarks evaluate this capability solely on numerical Big Five score prediction, leaving open whether models truly perceive personality through behavioral understanding or merely prejudge through superficial pattern matching. We address this gap with three contributions. (i) A new task: we formalize Grounded Personality Reasoning (GPR), which requires MLLMs to anchor each Big Five rating in observable evidence through a chain of rating, reasoning, and grounding. (ii) A new dataset: we release MM-OCEAN (1,104 videos, 5,320 MCQs), produced by a multi-agent pipeline with human verification, with timestamped behavioral observations, evidence-grounded trait analyses, and seven categories of cue-grounding MCQs. (iii) Benchmark and analysis: we design a three-tier evaluation (rating, reasoning, grounding) plus four sample-level failure-mode metrics: Prejudice Rate (PR), Confabulation Rate (CR), Integration-failure Rate (IR), and Holistic-grounding Rate (HR), and benchmark 27 MLLMs (13 closed, 14 open). The analysis uncovers a striking Prejudice Gap: across the field, 51% of correct ratings are not grounded in retrieved cues, and the Holistic-Grounding Rate spans only 0-33.5%. These findings expose a disconnect between getting the right score and reasoning for the right reason, charting a roadmap for grounded social cognition in MLLMs.
Abstract:In The Algebraic Mind, Gary Marcus identified three components essential for any adequate cognitive architecture: operations over variables, recursively structured representations, and a distinction between mental representations of individuals and kinds. He argued that standard multilayer perceptrons supported none of these, acknowledging that a neural implementation using registers and treelets, constructed via developmental programs rather than gradient descent, remained a programmatic conjecture. Twenty-five years later, the required substrate is now available. Our newly developed PyVaCoAl/VaCoAl is a hyperdimensional computing architecture organized end-to-end around a single algebraic primitive: XOR-and-shift over GF(2), implemented by primitive-polynomial linear-feedback shift registers. The architecture supports reversible variable binding via Bind(R,F) = R XOR shift(F), non-commutative compositional bundling that distinguishes "the dog bites the man" from "the man bites the dog," and address-space individual/kind separation under the same algebra. A companion perspective argues that the dentate gyrus-CA3 circuit is a biological homologue of this same engine, with developmentally specified mossy-fiber targeting supplying the innate microcircuitry Marcus anticipated. In this paper, we map the correspondence between Marcus's three pillars and the operational commitments of PyVaCoAl/VaCoAl. We reinterpret the treelet as an algebraic register set indexed by a primitive generator polynomial, arguing that this architecture provides a functional neural substrate meeting Marcus's specifications far more closely than the tensor products, circular convolution, or temporal synchrony available in 2001. We also demonstrate how this substrate naturally extends to Pearl's rung-3 counterfactual reasoning, a capability the original treelet program did not directly target.