Alert button
Picture for Hidenori Tanaka

Hidenori Tanaka

Alert button

Towards an Understanding of Stepwise Inference in Transformers: A Synthetic Graph Navigation Model

Add code
Bookmark button
Alert button
Feb 12, 2024
Mikail Khona, Maya Okawa, Jan Hula, Rahul Ramesh, Kento Nishi, Robert Dick, Ekdeep Singh Lubana, Hidenori Tanaka

Viaarxiv icon

How Capable Can a Transformer Become? A Study on Synthetic, Interpretable Tasks

Add code
Bookmark button
Alert button
Nov 21, 2023
Rahul Ramesh, Mikail Khona, Robert P. Dick, Hidenori Tanaka, Ekdeep Singh Lubana

Viaarxiv icon

Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks

Add code
Bookmark button
Alert button
Nov 21, 2023
Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Edward Grefenstette, Tim Rocktäschel, David Scott Krueger

Viaarxiv icon

In-Context Learning Dynamics with Random Binary Sequences

Add code
Bookmark button
Alert button
Oct 26, 2023
Eric J. Bigelow, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tomer D. Ullman

Figure 1 for In-Context Learning Dynamics with Random Binary Sequences
Figure 2 for In-Context Learning Dynamics with Random Binary Sequences
Figure 3 for In-Context Learning Dynamics with Random Binary Sequences
Figure 4 for In-Context Learning Dynamics with Random Binary Sequences
Viaarxiv icon

Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task

Add code
Bookmark button
Alert button
Oct 13, 2023
Maya Okawa, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka

Figure 1 for Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
Figure 2 for Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
Figure 3 for Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
Figure 4 for Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
Viaarxiv icon

Mechanistic Mode Connectivity

Add code
Bookmark button
Alert button
Nov 15, 2022
Ekdeep Singh Lubana, Eric J. Bigelow, Robert P. Dick, David Krueger, Hidenori Tanaka

Figure 1 for Mechanistic Mode Connectivity
Figure 2 for Mechanistic Mode Connectivity
Figure 3 for Mechanistic Mode Connectivity
Figure 4 for Mechanistic Mode Connectivity
Viaarxiv icon

What shapes the loss landscape of self-supervised learning?

Add code
Bookmark button
Alert button
Oct 02, 2022
Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda, Hidenori Tanaka

Figure 1 for What shapes the loss landscape of self-supervised learning?
Figure 2 for What shapes the loss landscape of self-supervised learning?
Figure 3 for What shapes the loss landscape of self-supervised learning?
Figure 4 for What shapes the loss landscape of self-supervised learning?
Viaarxiv icon

Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion

Add code
Bookmark button
Alert button
Jul 19, 2021
Daniel Kunin, Javier Sagastuy-Brena, Lauren Gillespie, Eshed Margalit, Hidenori Tanaka, Surya Ganguli, Daniel L. K. Yamins

Figure 1 for Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion
Figure 2 for Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion
Figure 3 for Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion
Figure 4 for Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion
Viaarxiv icon

Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning

Add code
Bookmark button
Alert button
Jul 08, 2021
Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka

Figure 1 for Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning
Figure 2 for Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning
Figure 3 for Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning
Figure 4 for Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning
Viaarxiv icon

Noether's Learning Dynamics: The Role of Kinetic Symmetry Breaking in Deep Learning

Add code
Bookmark button
Alert button
May 06, 2021
Hidenori Tanaka, Daniel Kunin

Figure 1 for Noether's Learning Dynamics: The Role of Kinetic Symmetry Breaking in Deep Learning
Figure 2 for Noether's Learning Dynamics: The Role of Kinetic Symmetry Breaking in Deep Learning
Viaarxiv icon