A cost function is calculated from Loss Functions , often the average of loss functions over an entire training data. A cost function itself is a type of objective function, which is object of our optimization.