Description

Presence Penalty is a parameter used in language models to reduce repetition in generated text. Unlike frequency penalty, which considers how often a token has been used, presence penalty applies a fixed penalty to all tokens that are already present in the generated text, regardless of their frequency. This encourages the model to introduce new concepts and vocabulary, potentially leading to more diverse and engaging output.

Examples

• 📚 Creative writing
• 🤖 Open-ended chatbot responses
• 📝 Brainstorming sessions

Applications