Yoshua Bengio
Turing Award Laureate | Scientific Director, Mila | Full Professor, Université de Montréal
Turing Award Laureate (2018) and Scientific Director of Mila - Quebec AI Institute, the world's largest academic research center for deep learning. Full Professor at Université de Montréal and recipient of the Order of Canada. Renowned for neural language models, attention mechanisms, Generative Adversarial Networks (GANs), and GFlowNets. Currently chairs the International Scientific Report on Advanced AI Safety.
Areas of focus
Professional niches
Proof of Work
Neural Machine Translation by Jointly Learning to Align and Translate (Additive Attention)
Co-authored the milestone paper introducing the attention mechanism into sequence-to-sequence translation, allowing decoders to dynamically attend to relevant input positions and directly inspiring the Transformer architecture.
Recurrent sequential processing precluded massive cluster-scale parallelization.
Context: Bidirectional RNN encoder with Bahdanau additive attention decoder on WMT14.
View missionA Neural Probabilistic Language Model
Introduced the seminal concept of learning continuous distributed word feature vectors alongside statistical language modeling, defeating the curse of dimensionality and establishing modern neural language modeling.
Computational bottleneck in softmax normalization over large vocabularies prior to hierarchical softmax and negative sampling.
Context: Feed-forward neural network sharing distributed representation layers across context words.
View mission