embeddings

For example, if your inputs are images or audio recordings, Just wanted to add: and word/character/phrase embeddings.

You train the embeddings on a large corpus and give them as part of the input.

Both of these ideas (word2vec and DeepWalk) are largely based on the idea of low level embeddings as you mention.

A great paper on learning embeddings for sparse representations is a paper by Yoshua Bengio et.

Word2vec maps words and their context with 2 embeddings, a dot product and a sigmoid.

There are other algorithms as well such as glove[1], document embeddings[2] and backpropagation based methods[3].

It uses a class of RNN they call a MemNN to do something that looks a lot like inference using phrase embeddings.

The connection between the random walk and embeddings has been characterized in [1] (disclosure: I am an author).

I'm not an expert, but I think this places strong constraints on the embeddings you can use.

Remarkably, in order to stay interpretable, word embeddings have to be shallow.

In C++ I am still tweaking the abstractions for the basic stuff (word embeddings, transition systems).

Second of all, SVM does not create any feature space (i.e., embeddings).

But one of the listed papers is also by Kapathy ("Deep Fragment Embeddings for Bidirectional Image Sentence Mapping"), and I think this might be what nl is complimenting as being done quickly.