AI glossary · Building and running AI
Inference
Inference is when a trained AI model is used to produce an answer, such as a chatbot writing a reply, as opposed to the training stage when it learns.
In more detail
Training happens once (or occasionally) and is very expensive. Inference happens every time someone uses the model, so its speed and cost determine how fast and affordable an AI product is.
Reasoning models use more inference computing per question, which is one reason they are slower. Running inference on your own device is local AI.
Related terms
AI modelAn AI model is the trained program at the heart of an AI tool: it takes an input, such as a question or a photo, and produces an output.GPUA GPU (graphics processing unit) is a chip that runs many calculations at once. Built for graphics, GPUs are now the main hardware for training and running AI.Local AILocal AI means running an AI model on your own phone or computer instead of on a company’s servers, so it can work offline and keep your data on your device.QuantizationQuantization shrinks an AI model by storing its numbers with less precision, so it uses less memory and runs faster, with a small loss in quality.
Browse related tools
AI tools listed on BroUseAI in these categories.
