What is cross-entropy loss and how does it differ from hinge loss?

• A. Both are identical.
• B. Cross-entropy measures the negative log-likelihood; hinge loss focuses on margin; cross-entropy is differentiable with probabilistic outputs.
• C. Cross-entropy is margin-based; hinge is log-likelihood.
• D. Cross-entropy is non-differentiable.

Answer: (B)

Cross-entropy loss measures how well the model’s predicted probability distribution over classes matches the true labels by taking the negative log-likelihood of the correct class. When a model outputs probabilities through softmax (for multiclass) or sigmoid (for binary), cross-entropy punishes confident but incorrect predictions more heavily and nudges the predicted probabilities to match the observed outcomes. It is differentiable with respect to the model’s parameters, which makes it ideal for gradient-based optimization.

Hinge loss, on the other hand, is about margins. It considers the score of the correct class and requires it to exceed the scores of other classes by at least a certain margin (usually 1). It penalizes only when this margin is violated, and it does not produce calibrated probabilities. It is not differentiable at the exact margin point (though subgradients exist there).

So the statement that cross-entropy corresponds to negative log-likelihood, hinge loss emphasizes margins, and cross-entropy remains differentiable with probabilistic outputs captures the key distinction, making it the best answer.