FastPrepImplement Multi-Head Attention
Problems/Implement Multi-Head Attention

Implement Multi-Head Attention

Apple logoApple● HardNEW GRADFULLTIMEPHONE SCREEN
QuestionFunctionExamplesConstraints

Problem statement

Source note: The earlier report explicitly asks for multi-head attention but omits tensor shapes and layer details; the judged core task matches that report at about 60%. Another report only names attention, so it does not confirm multiple heads, masks or projection details. Those specifics follow this practice contract. Neither report confirms that AI tools were allowed.

Implement a projection-free multi-head scaled dot-product attention forward pass. The inputs query, key and value are already projected row-major integer matrices. Split every row into heads contiguous slices of width w = d / heads.

For each query row and each head:

  1. Take the dot product of its head slice with each key row's matching slice.
  2. Divide each score by sqrt(w).
  3. Apply softmax across key rows: p[j] = exp(score[j]) / sum(exp(score)). Subtract the maximum score before exponentiation for numerical stability.
  4. Use those probabilities to take a weighted sum of the matching value slices.

Concatenate the head outputs in head order and return a matrix with one row per query and width d. No learned projections, masks, biases, dropout, backward pass or output projection are part of this exercise. Floating-point answers use a relative-or-absolute tolerance of 10^-6.

Function

multiHeadAttention(query: int[][], key: int[][], value: int[][], heads: int) → double[][]

Examples

Example 1

query = [[1]]key = [[0],[1]]value = [[10],[20]]heads = 1return = [[17.31058578630005]]

The scaled scores are 0 and 1. Their softmax weights are 1/(1+e) and e/(1+e), giving approximately 17.310586.

Example 2

query = [[1,0]]key = [[1,0],[0,1]]value = [[4,8],[6,2]]heads = 2return = [[4.53788284273999,5.0]]

The first head uses scores 1 and 0, giving approximately 4.537883. The second head has equal scores, so its output is the average of 8 and 2: 5.

Constraints

  • 1 <= query.length, key.length, value.length <= 20
  • Each matrix row contains 1 to 12 entries.
  • -20 <= every matrix entry <= 20
  • 1 <= heads <= 12
  • Every row in all three matrices has the same width d, and heads divides d.
  • key.length == value.length

More Apple problems

  • Evaluate DivisionPHONE SCREEN · Seen Nov 2026
  • Reverse a Linked ListPHONE SCREEN · Seen Nov 2026
  • Validate a BST with a Custom ComparisonPHONE SCREEN · Seen Nov 2026
All Apple interview questionsSee Apple hiring insights
Saved
public double[][] multiHeadAttention(int[][] query, int[][] key, int[][] value, int heads) {
    // Write your code here.
}
Checking account…