Alert button

Aggregated Learning: A Vector Quantization Approach to Learning with Neural Networks

Add code
Bookmark button
Alert button
Aug 15, 2018
Hongyu Guo, Yongyi Mao, Richong Zhang

Figure 1 for Aggregated Learning: A Vector Quantization Approach to Learning with Neural Networks
Figure 2 for Aggregated Learning: A Vector Quantization Approach to Learning with Neural Networks
Figure 3 for Aggregated Learning: A Vector Quantization Approach to Learning with Neural Networks
Figure 4 for Aggregated Learning: A Vector Quantization Approach to Learning with Neural Networks

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

We establish an equivalence between information bottleneck (IB) learning and an unconventional quantization problem, `IB quantization'. Under this equivalence, standard neural network models correspond to scalar IB quantizers. We prove a coding theorem for IB quantization, which implies that scalar IB quantizers are in general inferior to vector IB quantizers. This inspires us to develop a learning framework for neural networks, AgrLearn, that corresponds to vector IB quantizers. We experimentally verify that AgrLearn applied to some deep network models of current art improves upon them, while requiring less training data. With a heuristic smoothing, AgrLearn further improves its performance, resulting in new state of the art in image classification on Cifar10.

View paper onarxiv icon

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon