bengio

A great paper on learning embeddings for sparse representations is a paper by Yoshua Bengio et.

Yoshua Bengio will point out that there are functions that require exponentially more gates to encode when using shallower circuits.

People like Hinton, LeCun and Bengio have been working in the field for decades.

Hinton, Bengio or LeCun), or even just skim through the Wikipedia entry [2].

And Bengio and some other well-known people from the Deep Learning community.

The original paper on word embeddings from 2001 from Bengio et.

It is always impressive to me how both Professor LeCun, Bengio & Hinton stuck to their guns and worked on these problems while others were not so interested.

WRT the Glorot and Bengio paper, it’s true that there was a lot of excitement surrounding unsupervised pre-training of DNNs, but this mostly preceded the current wave of successes.

As the poster in that thread mentioned, much of the theory about using neural networks for NLP tasks was laid out pretty clearly in a paper by Yoshua Bengio from 2003 [2].

Yoshua Bengio has been working on neural nets for years and years and he leads the only one of the big three deep learning labs to not have a lot of commercial entanglements.

In particular LeCun and Bengio are in part responsible for the recent renewed interested in it ("Deep Learning").

Yoshua Bengio has touched on this with regards to depth from the point of view of circuit theory, but the argument is basically: "here's a couple circuits which are more efficiently represented by increased depth, therefore deep = good always".

Actually, Ilya explains his statement regarding minimizing training errors in his comment exchange with Bengio: "Although I didn't define it in the article, generalization (to me) means that the gap between the training and the test error is small.

So far they've had interviews with many of the big names in ML: Yann LeCun, Goeff Hinton, Yoshua Bengio.

Notice too that all 3 (Hinton, LeCun, and Bengio) worked directly together at some point, and collaborated often beyond that.

The most prominent I'm aware of saying thing to this effect are Yoshua Bengio, Geoff Hinton, and Yann LeCun.