Gradient Descent Linear Regression Predictions
Problem statement
Implement batch gradient descent for multivariable linear regression from scratch, without a machine-learning or numerical library. Train on xTrain and yTrain, then return one predicted value for each row of xTest in its original order.
Each training row has m features. The model is prediction = b + theta[0] * x[0] + ... + theta[m - 1] * x[m - 1].
Training rule
Start with bias initialBias and coefficients initialThetas. Perform exactly iterations full-batch updates. In one update:
- Use the current parameters to predict every training label. Define
error[i] = prediction[i] - yTrain[i]. - Compute the mean-squared-error gradients:
db = (2 / n) * sum(error[i])anddTheta[j] = (2 / n) * sum(error[i] * xTrain[i][j]). - Update the bias and every coefficient simultaneously:
b = b - learningRate * dbandtheta[j] = theta[j] - learningRate * dTheta[j]. Every gradient in this step uses the parameters from before the step.
After the final update, evaluate the trained model on xTest and return those predictions. Do not replace the required updates with a closed-form regression fit or stop early.
Practice contract
The initial parameters are supplied to represent a reproducible realization of random initialization. Generate no additional random values. When iterations is zero, predict using the supplied parameters. An empty xTest produces an empty result. Use double-precision arithmetic; outputs are compared with tolerance 10^-6.
Function
predictLinearRegression(xTrain: double[][], yTrain: double[], xTest: double[][], initialBias: double, initialThetas: double[], learningRate: double, iterations: int) → double[]Examples
Example 1
xTrain = [[0],[1]]yTrain = [1,3]xTest = [[0],[0.5],[1]]initialBias = 0initialThetas = [0]learningRate = 0.05iterations = 1return = [0.2,0.275,0.35]The initial predictions are [0, 0]. The mean-squared-error gradients are db = -4 and dTheta = -3. One simultaneous update produces b = 0.2 and theta = [0.15], so the three test predictions are [0.2, 0.275, 0.35].
Example 2
xTrain = [[1,0],[0,1]]yTrain = [3,0]xTest = [[0.5,0.5],[-1,1]]initialBias = 1initialThetas = [2,-1]learningRate = 0.025iterations = 5return = [1.5,-2.0]The supplied parameters already predict both training labels exactly, so every gradient is zero. The test predictions are 1 + 2(0.5) - 0.5 = 1.5 and 1 + 2(-1) - 1 = -2.
Constraints
- The following bounds are practice assumptions.
1 <= xTrain.length = yTrain.length <= 200.1 <= initialThetas.length <= 8.0 <= xTest.length <= 200.- Every training and test row has exactly
initialThetas.lengthfeatures. - Feature values lie in
[-1, 1]. Training labels, the initial bias, and initial coefficients lie in[-10, 10]. 0 < learningRate <= 0.05and0 <= iterations <= 1000.- All numeric inputs are finite. Training rows are processed in their given order.