Group-Relative Policy Advantages
Amazon● MediumFULLTIMEONSITE INTERVIEWProblem statement
Compute group-relative advantages for sampled policy responses. Response i belongs to groupIds[i] and receives the integer reward correctnessRewards[i] + formatRewards[i].
Within each group, compute the population mean and population standard deviation of the combined rewards. The advantage of a response is (reward - mean) / standardDeviation. When a group's standard deviation is zero, every response in that group has advantage zero.
Return the advantages in original response order, formatted to exactly six decimal places.
Function
groupRelativeAdvantages(groupIds: int[], correctnessRewards: int[], formatRewards: int[]) → String[]Examples
Example 1
groupIds = [0,0,1,1,1]correctnessRewards = [1,0,1,1,0]formatRewards = [1,0,0,1,0]return = ["1.000000","-1.000000","0.000000","1.224745","-1.224745"]Each prompt group is normalized independently after correctness and format rewards are combined.
Example 2
groupIds = [7,7]correctnessRewards = [1,1]formatRewards = [0,0]return = ["0.000000","0.000000"]Equal rewards have zero variance, so both normalized advantages are zero.
Constraints
2 <= groupIds.length <= 100000- All three arrays have the same length.
- Each group ID appears at least twice.
-100 <= correctnessRewards[i], formatRewards[i] <= 100