Gradient Descent Linear Regression from Scratch
Learn this problemProblem statement
Your task is to implement parts of the Gradient Descent optimization algorithm from scratch (i.e., without importing any libraries or packages). You will apply this algorithm for linear regression — finding the coefficients for the following equation:
y = b + θ₁x₁ + θ₂x₂ + ... + θₘxₘ
As a reminder, Gradient Descent comprises following steps:
- Randomly set initial values of bias
band thetasθ~i. - Calculate predicted value
ŷ. - Calculate partial derivative of a cost function with respect to bias.
d_b = ∂/∂b (1/n) Σᵢ₌₁ⁿ(yᵢ - ŷᵢ)² = -2/n Σᵢ₌₁ⁿ(yᵢ - ŷᵢ) - Calculate partial derivatives of a cost function with respect to thetas.
d_θₖ = ∂/∂θₖ (1/n) Σᵢ₌₁ⁿ(yᵢ - ŷᵢ)² = -2/n Σᵢ₌₁ⁿxᵢᵏ(yᵢ - ŷᵢ) - Update bias and thetas (
αis a learning rate).b = b - αd_bθₖ = θₖ - αd_θₖ - Repeat steps 2-5 for the
iterationstimes.
FastPrep execution adapter
The source image is cropped before the complete callable and output contract. For deterministic grading, FastPrep receives the initial bias and theta values that represent the source's randomly set initial state, then returns the final bias followed by the final coefficients. This adapter does not replace the source requirement to begin from initialized values.
Function
fitLinearRegression(xTrain: double[][], yTrain: double[], initialBias: double, initialThetas: double[], learningRate: double, iterations: int) → double[]Examples
Example 1
xTrain = [[1], [2]]yTrain = [3, 5]initialBias = 0initialThetas = [0]learningRate = 0.1iterations = 1return = [0.8, 1.3]FastPrep-authored runnable example (not shown in the source image): The initial predictions are both 0, so the residuals are -3 and -5. Therefore dBias = -8 and dTheta[0] = -13. One update with learning rate 0.1 produces bias 0.8 and coefficient 1.3.
Constraints
- FastPrep execution-adapter constraints (not shown in the source image):
xTrain.length == yTrain.lengthand the arrays are non-empty.- Every row of
xTrainhas exactlyinitialThetas.lengthfeatures. learningRate > 0anditerations >= 0.- All inputs and expected final parameters are finite numbers.