Picture for Kazuki Irie

Kazuki Irie

MoEUT: Mixture-of-Experts Universal Transformers

Add code
May 25, 2024
Viaarxiv icon

Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers

Add code
May 24, 2024
Viaarxiv icon

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention

Add code
Dec 14, 2023
Viaarxiv icon

Automating Continual Learning

Add code
Dec 01, 2023
Viaarxiv icon

Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions

Add code
Oct 24, 2023
Viaarxiv icon

Approximating Two-Layer Feedforward Networks for Efficient Transformers

Add code
Oct 23, 2023
Viaarxiv icon

Exploring the Promise and Limits of Real-Time Recurrent Learning

Add code
May 30, 2023
Figure 1 for Exploring the Promise and Limits of Real-Time Recurrent Learning
Figure 2 for Exploring the Promise and Limits of Real-Time Recurrent Learning
Figure 3 for Exploring the Promise and Limits of Real-Time Recurrent Learning
Figure 4 for Exploring the Promise and Limits of Real-Time Recurrent Learning
Viaarxiv icon

Mindstorms in Natural Language-Based Societies of Mind

May 26, 2023
Figure 1 for Mindstorms in Natural Language-Based Societies of Mind
Figure 2 for Mindstorms in Natural Language-Based Societies of Mind
Figure 3 for Mindstorms in Natural Language-Based Societies of Mind
Figure 4 for Mindstorms in Natural Language-Based Societies of Mind
Viaarxiv icon

Contrastive Training of Complex-Valued Autoencoders for Object Discovery

Add code
May 25, 2023
Figure 1 for Contrastive Training of Complex-Valued Autoencoders for Object Discovery
Figure 2 for Contrastive Training of Complex-Valued Autoencoders for Object Discovery
Figure 3 for Contrastive Training of Complex-Valued Autoencoders for Object Discovery
Figure 4 for Contrastive Training of Complex-Valued Autoencoders for Object Discovery
Viaarxiv icon

Accelerating Neural Self-Improvement via Bootstrapping

Add code
May 02, 2023
Figure 1 for Accelerating Neural Self-Improvement via Bootstrapping
Figure 2 for Accelerating Neural Self-Improvement via Bootstrapping
Figure 3 for Accelerating Neural Self-Improvement via Bootstrapping
Viaarxiv icon