Scaled Dot-Product Attention
Updated
Problem statement
Implement unmasked single-head scaled dot-product attention for three nonempty matrices:
querieshas shapen by d.keyshas shapem by d.valueshas shapem by v.
For each query row, compute one score per key as the dot product divided by sqrt(d). Apply maximum-subtracted softmax across that score row, then return the weighted sum of the value rows. The result has shape n by v.
There is no mask, batch dimension, learned projection, bias, or dropout. Implement the matrix loops, stable softmax, and weighted sum directly without a machine-learning or numerical-array library.
Function
scaledDotProductAttention(queries: double[][], keys: double[][], values: double[][]) → double[][]Examples
Example 1
queries = [[1.0,0.0]]keys = [[1.0,0.0],[0.0,1.0]]values = [[10.0,0.0],[0.0,20.0]]return = [[6.697615493266569,6.604769013466862]]The scaled scores are approximately [0.7071067812, 0]. Their softmax weights are approximately [0.6697615493, 0.3302384507], which weight the two value rows.
Example 2
queries = [[0.0,0.0]]keys = [[1.0,0.0],[-1.0,0.0]]values = [[2.0,4.0],[6.0,8.0]]return = [[4.0,6.0]]The zero query has equal score for both keys, so softmax gives each value row weight 0.5.
Example 3
queries = [[1.0,2.0],[-3.0,4.0]]keys = [[5.0,6.0]]values = [[7.0,-2.0,1.0]]return = [[7.0,-2.0,1.0],[7.0,-2.0,1.0]]With only one key, every query gives its value row softmax weight 1.
Constraints
1 <= n, m <= 201 <= d, v <= 10- Every matrix is rectangular and follows the stated compatible shapes.
-20 <= matrix entry <= 20- Every matrix entry is finite: no value is NaN or infinity.
- The result is compared entry by entry with absolute tolerance
1e-9.
More Amazon Phone Screen questions
- Number of IslandsPHONE SCREEN · ONSITE INTERVIEW · Seen Nov 2026
- 3SumPHONE SCREEN · Seen Nov 2026
- Circular Primes Below a LimitPHONE SCREEN · Seen Nov 2026
- Cheapest Flights Within K StopsPHONE SCREEN · ONSITE INTERVIEW · Seen Nov 2026
- Course SchedulePHONE SCREEN · ONSITE INTERVIEW · Seen Nov 2026
- Course Schedule IIPHONE SCREEN · ONSITE INTERVIEW · Seen Nov 2026
- LRU Cache for Query ResultsPHONE SCREEN · ONSITE INTERVIEW · Seen Nov 2026
- Lowest Common Ancestor in an N-ary TreePHONE SCREEN · ONSITE INTERVIEW · Seen Nov 2026
- Two SumOA · PHONE SCREEN · Seen Nov 2026
- Count the 1 Bits in a 32-Bit ValuePHONE SCREEN · Seen Nov 2026
Similar questions at other companies
- Implement Cross-Entropy LossOpenAI · PHONE SCREENOpenAI interview questions
- Vanilla RNN Hidden StatesNVIDIA · PHONE SCREENNVIDIA interview questions
- Transformer Attention and SwiGLU Forward PassGoogle · ONSITE INTERVIEWGoogle interview questions
- Deterministic K-Means AssignmentsLyft · PHONE SCREENLyft interview questions
- Deterministic K-Means AssignmentsMeta · PHONE SCREEN · ONSITE INTERVIEWMeta interview questions
- Zigzag Board Adjacent SwapsUber · PHONE SCREENUber interview questions
- Ultimate Tic-Tac-Toe Move SimulatorHebbia · PHONE SCREEN
- Deterministic K-Means AssignmentsMicrosoft · PHONE SCREEN · ONSITE INTERVIEWMicrosoft interview questions
- Implement K-Means ClusteringApple · PHONE SCREENApple interview questions
- Automation Pipeline, Part 1: Single WorkerPlaid · PHONE SCREENPlaid interview questions
Amazon interview experiences
- Amazon SWE OA with AI Coding: 1 Experience
- Amazon SWE Intern Phone Screen Interview Experience: 1 Experience
- Amazon New Grad SDE I OA and Virtual Onsite: 1 Experience
- Amazon BIE Intern SQL and Interview Questions: 17 Experiences
- Amazon Experienced SWE AI-Assisted Debugging OA: 1 Experience
- Amazon SDE II (L5) Interview Questions: 5 Experiences