Gradient (Calculus)

Gradient is the measure of change in output of function, for change in it's input, and is measured as a derivative of a function that has more than one input variable(slope).


Note:

  • The combination of all the derivatives for all the parameters is the loss gradient
  • A gradient measures how much the output of a function changes if you change the inputs a little bit.
  • gradient is well conditioned: it’s not radically larger in one direction than another.