AI glossary · Machine learning
Gradient descent
Gradient descent is the method most AI models use to learn: it repeatedly nudges the model’s settings in the direction that reduces its errors.
Also called stochastic gradient descent, SGD, optimisation algorithm
In more detail
Picture walking down a foggy hill by always stepping in the steepest downward direction. The “hill” is the loss function, and each step adjusts the model’s parameters slightly.
Stochastic gradient descent (SGD) takes each step using a small random batch of examples, which is much faster on large datasets. The size of each step is the learning rate, a hyperparameter.
Related terms
Loss functionA loss function measures how wrong an AI model’s predictions are during training. Training tries to make this number as small as possible.BackpropagationBackpropagation is how a neural network learns from mistakes: it works backwards from the error to find how much each connection caused it, then adjusts them.HyperparameterA hyperparameter is a setting chosen before training an AI model, such as how fast it learns or how many layers it has, that shapes how training goes.ParametersParameters are the internal numbers an AI model learns during training. A model’s size, such as “70 billion parameters”, counts them.
Browse related tools
AI tools listed on BroUseAI in these categories.