Logistic Regression with Gradient Descent
Updated
Problem statement
Train a binary logistic-regression classifier with gradient descent, without a machine-learning or automatic-differentiation library. features holds n samples with d real-valued features each, and labels[i] is the class (0 or 1) of sample i.
Model. The predicted probability of class 1 is p[i] = sigmoid(bias + sum_j(weights[j] * features[i][j])), where sigmoid(z) = 1 / (1 + e^(-z)).
Training. Minimize the mean binary cross-entropy. Start with every weight and the bias at 0.0 and perform exactly iterations full-batch updates. In each update, compute the gradient from all samples with the same current parameters:
gradWeight[j] = sum_i((p[i] - labels[i]) * features[i][j]) / ngradBias = sum_i(p[i] - labels[i]) / n
Then update every parameter at the same time with parameter = parameter - learningRate * gradient. Do not shuffle, use momentum, add regularization or stop early.
Return a double[] with the d final weights in feature order, followed by the final bias. Each value is accepted with absolute or relative error at most 10-6.
Function
trainLogisticRegression(features: double[][], labels: int[], learningRate: double, iterations: int) → double[]Examples
Example 1
features = [[1.0],[-1.0]]labels = [1,0]learningRate = 0.5iterations = 1return = [0.25,0.0]- All parameters start at 0, so both predictions are
sigmoid(0) = 0.5. The errors are0.5 - 1 = -0.5and0.5 - 0 = 0.5. - Weight gradient:
(-0.5 × 1 + 0.5 × (-1)) / 2 = -0.5, so the weight becomes0 - 0.5 × (-0.5) = 0.25. - Bias gradient:
(-0.5 + 0.5) / 2 = 0, so the bias stays0. The answer is[0.25, 0.0].
Example 2
features = [[0.5,-0.2],[0.1,0.4],[-0.3,0.9]]labels = [1,1,0]learningRate = 0.1iterations = 0return = [0.0,0.0,0.0]- With zero iterations nothing is updated, so every weight and the bias stay
0.0.
Example 3
features = [[0.2,0.8],[0.9,-0.4],[-0.6,0.1],[0.3,0.3]]labels = [1,1,0,0]learningRate = 0.3iterations = 2return = [0.103720492303,0.000167276229,-0.00078737613]- Two full-batch updates of both weights and the bias, each computed from all four samples with the same parameter snapshot; the result is the two weights followed by the bias.
Constraints
1 <= n <= 200and1 <= d <= 8.features.length == labels.length == n, and every feature row has exactlydentries.- All feature values lie in
[-1.0, 1.0]. - Each
labels[i]is0or1. 0 <= learningRate <= 1.0and0 <= iterations <= 200.
More Yahoo interview questions
Similar questions at other companies
- Constant-Time Weighted Pick with an Alias TableEtched · PHONE SCREEN
- Adaptive Calibration ErrorOpenAI · PHONE SCREENOpenAI interview questions
- Random Pick with Weight Using Recorded DrawsGoogle · PHONE SCREEN · ONSITE INTERVIEWGoogle interview questions
- Reachable Transfer PositionsCitadel · PHONE SCREENCitadel interview questions
- Find All Pythagorean TripletsDell Technologies · PHONE SCREENDell Technologies interview questions
- Sort a Quadratic-Transformed ArrayLinkedIn · PHONE SCREENLinkedIn interview questions
- Filter Paginated Medical RecordsNVIDIA · PHONE SCREENNVIDIA interview questions
- Minimum Operations to Make Every Array Element OneOtter.ai · PHONE SCREENOtter.ai interview questions
- Evenly Schedule Vehicle InspectionsAmazon · PHONE SCREEN
- Coin Game From The EndsTekion · PHONE SCREENTekion interview questions