AI glossary · Generative AI and LLMs
Mixture of experts (MoE)
Mixture of experts (MoE) is a model design that splits a large model into many specialised parts and only uses a few of them for each piece of text.
In more detail
A “router” inside the model sends each token to the most suitable expert sub-networks. Because only a fraction of the model is active at a time, it can hold a lot of knowledge while staying faster and cheaper to run.
Several well-known open and commercial models use this design.
Related terms
Large language model (LLM)A large language model (LLM) is an AI model trained on vast amounts of text to understand and generate language. It powers chatbots like ChatGPT.ParametersParameters are the internal numbers an AI model learns during training. A model’s size, such as “70 billion parameters”, counts them.InferenceInference is when a trained AI model is used to produce an answer, such as a chatbot writing a reply, as opposed to the training stage when it learns.
Browse related tools
AI tools listed on BroUseAI in these categories.
