Description

Context length, also known as context window or sequence length, refers to the maximum number of tokens that an AI model, particularly a language model, can process or consider at once. It determines how much previous information the model can use to understand the current input or generate the next output. Longer context lengths allow models to maintain coherence over larger pieces of text and handle more complex tasks, but they also require more computational resources. The context length is a key parameter that affects a model's performance and capabilities.

Examples

• 🔢 2048 tokens for GPT-3
• 🔢 4096 tokens for GPT-4
• 📊 Varying lengths for different model versions

Applications