AI glossary · Building and running AI
Quantization
Quantization shrinks an AI model by storing its numbers with less precision, so it uses less memory and runs faster, with a small loss in quality.
Also called quantisation, model compression
In more detail
A model’s parameters are normally stored as detailed decimal numbers. Rounding them to simpler formats can cut the model’s size by half or more.
Quantization is a key reason large models can run on laptops and phones (see local AI).
Related terms
Local AILocal AI means running an AI model on your own phone or computer instead of on a company’s servers, so it can work offline and keep your data on your device.ParametersParameters are the internal numbers an AI model learns during training. A model’s size, such as “70 billion parameters”, counts them.DistillationDistillation trains a smaller AI model to copy the behaviour of a larger one, so you get much of the quality at a fraction of the size and cost.InferenceInference is when a trained AI model is used to produce an answer, such as a chatbot writing a reply, as opposed to the training stage when it learns.
Browse related tools
AI tools listed on BroUseAI in these categories.
