How should I practice Multiclass Softmax Loss and Gradient?

Start by reading the full statement, solve the visible examples, then compare the pattern against related Amazon phone screen problems in FastPrep.

Is Multiclass Softmax Loss and Gradient associated with Amazon?

FastPrep tags this as a Amazon phone screen coding practice problem when the problem catalog has that company and stage context.

Multiclass Softmax Loss and Gradient

Amazon● MediumFULLTIMEPHONE SCREEN

Learn

Problem statement

For this exercise, assume a batch of classification examples and the current parameters of a linear model. Implement its loss and analytic derivatives directly, without an automatic-differentiation or machine-learning library.

features is an n-by-d matrix; labels has one value per sample.

There are c classes. The weight row weights[k] and biases[k] belong to class k. For sample i, its class logit is z[i][k] = biases[k] + sum_j(features[i][j] * weights[k][j]). Probabilities are the softmax of that sample's logits. The loss is mean multiclass cross-entropy: L = -sum_i(ln(p[i][labels[i]])) / n.

Return a flat array of length 1 + c*d + c in this exact order: the loss, all weight derivatives in class-major order (every feature of class 0, then class 1, and so on), then the c bias derivatives. In particular, result[1+k*d+j] = dL/dweights[k][j] and result[1+c*d+k] = dL/dbiases[k]. Do not update parameters. Evaluate loss and probabilities stably from logits.

No regularization, probability clipping, class weighting or label smoothing is used; each label is a zero-based class index.

Function

multiclassLossGradient(features: double[][], labels: int[], weights: double[][], biases: double[]) → double[]

Examples

Example 1

features = [[1.0]]labels = [0]weights = [[0.0],[0.0]]biases = [0.0,0.0]return = [0.6931471805599453,-0.5,0.5,-0.5,0.5]

Two equal logits give loss log(2), weight derivatives -0.5 and 0.5, and the same two bias derivatives.

Example 2

features = [[0.0,0.0]]labels = [2]weights = [[0.0,0.0],[0.0,0.0],[0.0,0.0]]biases = [0.0,0.0,0.0]return = [1.0986122886681098,0.0,0.0,0.0,0.0,0.0,0.0,0.3333333333333333,0.3333333333333333,-0.6666666666666666]

All probabilities are one third. All six weight derivatives are zero; bias derivatives are one third, one third and minus two thirds.

Example 3

features = [[1.0],[1.0]]labels = [0,1]weights = [[0.0],[0.0]]biases = [0.0,0.0]return = [0.6931471805599453,0.0,0.0,0.0,0.0]

Equal features and balanced labels cancel every derivative while the mean loss remains log(2).

Constraints

1 <= n <= 1000 and 1 <= d <= 50.
All arrays have the compatible shapes described above; all input numbers are finite.
Feature values, weights and biases are in [-10,10].
2 <= c <= 10 and 0 <= labels[i] < c.
Each returned number is compared with absolute or relative tolerance 10^-6.
Use analytic derivatives; finite-difference estimates are outside the exercise contract.