Dzmitry Bahdanau

Dzmitry Bahdanau

A Belarusian researcher who, with Kyunghyun Cho and Yoshua Bengio, introduced the attention mechanism in 2014. Machine translation systems at the time squeezed an entire source sentence into one fixed-length vector; their model instead learned to look back over the whole input and weight the words most relevant to each output word. Attention removed the bottleneck, improved long sentences dramatically, and three years later became the sole ingredient of the Transformer. (See also: Kyunghyun Cho, Yoshua Bengio, Transformer, Context window)