AI glossary · Generative AI and LLMs
Transformer
A transformer is the neural network design behind almost every modern language model. It reads a whole passage at once and works out how its words relate.
Also called transformer model, transformer architecture
In more detail
Transformers were introduced by Google researchers in the 2017 paper “Attention Is All You Need”. Their key idea is the attention mechanism, which lets the model weigh which words in a passage matter most to each other.
Because transformers process text in parallel instead of word by word, they can be trained on far more data than older designs like recurrent neural networks. That made today’s LLMs possible. Transformers are now also used for images, audio and video.
Example
In “The cat didn’t cross the road because it was tired”, attention helps the model link “it” to “the cat”, not “the road”.
Related terms
Browse related tools
AI tools listed on BroUseAI in these categories.
