vaswani

The Vaswani paper came out 8 years ago so implementations will be pretty highly optimized at this point.

If anything OpenAI was dependent on the transformers paper from Google from Ashish Vaswani and others.

Adept still hasn't released a product and its $350m Series B was only in March, shortly after Vaswani and Parmar exited.

I think if you know some linear algebra and read Vaswani et al.

These are all clear improvements to the Vaswani architecture.

I was just aware of the appearance of the Vaswani paper, but only now have I returned to the subject in a way that requires more rigour.

Attention is All You Need - which is a paper published by Vaswani et al., very interesting publication that is a key for understanding the self-attention mechanism and how it powers modern NLP models like GPT.

It'll be Vaswani and the others for the transformer, then maybe Zelikman and those on that paper for thought tokens, then maybe some of the RNN people and word embedding people will be cited as pioneers.

Or is my presumptuous chain of logic just a random thoughtless unconscious probabilistic stream of tokens that has nothing to do with reality, and I don't realize I'm speaking to Ashish Vaswani himself?

I won't speak to the motivations or views of the specific authors of Vaswani et al, but there's a long history, both distant and recent, of drawing connections between information theory, compression, prediction, and intelligence, including in the context of language modeling.

We raised a $33m Series A from Geoff Hinton’s Radical Ventures, and are backed by some of the best minds in AI, from Ashish Vaswani (Attention is All You Need), Oriol Vinyals (Deepmind), Christian Szegedy (X.ai) and the AI leadership at Twitter, Amazon, Isomorphic, and Google.

V7, UK HQ, recently raised a $33m Series A from Geoff Hinton’s (Inventor of AI as we know it) Radical Ventures, and are backed by some of the best minds in AI, from Ashish Vaswani (Invented Transformers, the basis of GPT) Oriol Vinyals (Google Deepmind and Gemini co-lead), Christian Szegedy (X.ai) and AI leadership at Amazon, X, Isomorphic, and Google.

"As [Essential AI Labs (founded in 2021 by Vaswani)] changes focus, Vaswani is asking investors for at least $150 million." expected, yet still funny.

This was in fact the trend for a century or more (see inventions ranging from the polio vaccine to "Attention is all you need" by Vaswani et.

As a follow up just to refresh your memory: “Attention Is All You Need” (Vaswani et al., 2017) Length: 11 pages of main content, 5 pages of references and appendix 2.

The issue is that Vaswani, et al never mentions this relationship.

That part is no longer up for debate by anyone who's been paying attention since AlphaGo, never mind Vaswani.

Transformers for language modeling (Vaswani et al., 2023).