Alert button
Picture for Adhiguna Kuncoro

Adhiguna Kuncoro

Alert button

DiLoCo: Distributed Low-Communication Training of Language Models

Nov 14, 2023
Arthur Douillard, Qixuan Feng, Andrei A. Rusu, Rachita Chhaparia, Yani Donchev, Adhiguna Kuncoro, Marc'Aurelio Ranzato, Arthur Szlam, Jiajun Shen

Viaarxiv icon

On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research

Jun 05, 2023
Made Nindyatama Nityasya, Haryo Akbarianto Wibowo, Alham Fikri Aji, Genta Indra Winata, Radityo Eko Prasojo, Phil Blunsom, Adhiguna Kuncoro

Figure 1 for On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research
Figure 2 for On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research
Figure 3 for On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research
Figure 4 for On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research
Viaarxiv icon

A Natural Bias for Language Generation Models

Dec 19, 2022
Clara Meister, Wojciech Stokowiec, Tiago Pimentel, Lei Yu, Laura Rimell, Adhiguna Kuncoro

Figure 1 for A Natural Bias for Language Generation Models
Figure 2 for A Natural Bias for Language Generation Models
Figure 3 for A Natural Bias for Language Generation Models
Figure 4 for A Natural Bias for Language Generation Models
Viaarxiv icon

Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale

Mar 01, 2022
Laurent Sartran, Samuel Barrett, Adhiguna Kuncoro, Miloš Stanojević, Phil Blunsom, Chris Dyer

Figure 1 for Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale
Figure 2 for Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale
Figure 3 for Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale
Figure 4 for Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale
Viaarxiv icon

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

Dec 08, 2021
Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d'Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew Johnson, Blake Hechtman, Laura Weidinger, Iason Gabriel, William Isaac, Ed Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, Geoffrey Irving

Figure 1 for Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Figure 2 for Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Figure 3 for Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Figure 4 for Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Viaarxiv icon

IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation

Apr 16, 2021
Samuel Cahyawijaya, Genta Indra Winata, Bryan Wilie, Karissa Vincentio, Xiaohong Li, Adhiguna Kuncoro, Sebastian Ruder, Zhi Yuan Lim, Syafri Bahar, Masayu Leylia Khodra, Ayu Purwarianti, Pascale Fung

Figure 1 for IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation
Figure 2 for IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation
Figure 3 for IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation
Figure 4 for IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation
Viaarxiv icon

Pitfalls of Static Language Modelling

Feb 03, 2021
Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Sebastian Ruder, Dani Yogatama, Kris Cao, Tomas Kocisky, Susannah Young, Phil Blunsom

Figure 1 for Pitfalls of Static Language Modelling
Figure 2 for Pitfalls of Static Language Modelling
Figure 3 for Pitfalls of Static Language Modelling
Figure 4 for Pitfalls of Static Language Modelling
Viaarxiv icon

Syntactic Structure Distillation Pretraining For Bidirectional Encoders

May 27, 2020
Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried, Dani Yogatama, Laura Rimell, Chris Dyer, Phil Blunsom

Figure 1 for Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Figure 2 for Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Figure 3 for Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Figure 4 for Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Viaarxiv icon

Scalable Syntax-Aware Language Models Using Knowledge Distillation

Jun 14, 2019
Adhiguna Kuncoro, Chris Dyer, Laura Rimell, Stephen Clark, Phil Blunsom

Figure 1 for Scalable Syntax-Aware Language Models Using Knowledge Distillation
Figure 2 for Scalable Syntax-Aware Language Models Using Knowledge Distillation
Figure 3 for Scalable Syntax-Aware Language Models Using Knowledge Distillation
Figure 4 for Scalable Syntax-Aware Language Models Using Knowledge Distillation
Viaarxiv icon

Unsupervised Recurrent Neural Network Grammars

Apr 15, 2019
Yoon Kim, Alexander M. Rush, Lei Yu, Adhiguna Kuncoro, Chris Dyer, Gábor Melis

Figure 1 for Unsupervised Recurrent Neural Network Grammars
Figure 2 for Unsupervised Recurrent Neural Network Grammars
Figure 3 for Unsupervised Recurrent Neural Network Grammars
Figure 4 for Unsupervised Recurrent Neural Network Grammars
Viaarxiv icon