AI glossary · Machine learning
Attention mechanism
An attention mechanism lets an AI model focus on the most relevant parts of its input, such as which earlier words matter for understanding the current one.
Also called attention, self-attention
In more detail
For each word, attention scores how much every other word should influence it. This lets models connect ideas far apart in a text, like a pronoun and the name it refers to.
Self-attention is the core of the transformer design that powers modern large language models.
Related terms
TransformerA transformer is the neural network design behind almost every modern language model. It reads a whole passage at once and works out how its words relate.Large language model (LLM)A large language model (LLM) is an AI model trained on vast amounts of text to understand and generate language. It powers chatbots like ChatGPT.Context windowThe context window is how much text an AI model can consider at once, including your messages, files and its own replies, measured in tokens.Recurrent neural network (RNN)A recurrent neural network (RNN) is a type of neural network that reads data in sequence, one step at a time, keeping a memory of what came before.
Browse related tools
AI tools listed on BroUseAI in these categories.
