Problem · Array
Scaled Dot-Product Attention
Problem statement
Implement unmasked single-head scaled dot-product attention for three nonempty matrices:
querieshas shapen by d.keyshas shapem by d.valueshas shapem by v.
For each query row, compute one score per key as the dot product divided by sqrt(d). Apply maximum-subtracted softmax across that score row, then return the weighted sum of the value rows. The result has shape n by v.
The problem statement continues
ProExamples
Example 1
queries = [[1.0,0.0]]keys = [[1.0,0.0],[0.0,1.0]]values = [[10.0,0.0],[0.0,20.0]]return = [[6.697615493266569,6.604769013466862]]The scaled scores are approximately [0.7071067812, 0]. Their softmax weights are approximately [0.6697615493, 0.3302384507], which weight the two value rows.
FastPrep Pro
Reported in 1 Amazon interview this weekUnlock this recently reported problem
FastPrep Pro gives you full access to interview problems reported within the last week.
- Full problem statement and constraints
- 2 more worked examples, explained
- Guided hints and editorial
- Run your code on real test cases
$9/month
Pro subscription, billed yearly — or $19 month-to-month. Cancel anytime.
Free plan — 2 of 2 free unlocks used this week