autoencoders

The original paper found that nets pretrained as autoencoders were the least susceptible to it, but still failed.

Geoff Hinton popularized pre-training nets with unsupervised latent-feature learning autoencoders in the 90s, making training nets much faster.

If you are trying to do feature learning, you are using RBMs, Denoising AutoEncoders and you will use reconstruction entropy.

This was also hinted by AutoEncoders where pieces of a net are built separately bottom-up.

Convolutional networks were also developed in the 80s as well as backpropagating algorithms (autoencoders).

Curious if you think the low-level features learned from the vector_exp latent factors are different from, say, unsupervised learning with sparse autoencoders?

Or look for deep learning, autoencoders, hierarchical hidden markov models, hierarchical temporal memory.

This is the basis of the work around autoencoders, deep belief networks, and it guarantees that you can tractably sample from your latent representation.

Computer clusters of hundreds and thousands of nodes are used to run SGD at high speeds with the above techniques, and pretraining with autoencoders isn't quite necessary anymore.

> Autoencoders are an extremely exciting new approach to unsupervised learning They are not new.

After all, it's only recently that we figured out how to efficiently train deep nets (either pre-training with Autoencoders or RBMs, or through Hessian-free optimization).

For denoising autoencoders, watch your corruption level.

Deep autoencoders[1] are allowing for computer representations of "similar" concepts.

Now, as far as the difference between PCA and eg an autoencoder (which is similar to an RBM, but different--both RBMs and autoencoders are "deep neural networks")--well, PCA just learns a linear mapping.

If you rephrase your question to "What happens when I add random noise to the inputs of a neural network and try to teach it to output a "denoised" version of the input." and you've just invented denoising autoencoders.

The idea of ANN undisputedly inspired from research investigating how brain actually works, and now it has set the state-of-the-art on many areas it has been applied, but to do this we had to resort to "ugly hacks" in the perspective of "the algorithm", to name some of them would be Autoencoders and Restricted Boltzmann Machines.

Deep Convolutional Neural Networks, various deep Autoencoders, etc.).