Dog Retrieval Precision and Recall
Problem statement
A corpus contains exactly ten image files, identified by 0 through 9. The entry labels[i] is 1 when file i depicts a dog and 0 for another animal. A retrieval API was asked for k files, and its received response is represented by returnedIds and responsePresent.
Return [precision, recall] using these rules:
- If
responsePresentisfalse, the response isNone. IgnorereturnedIdsand return[-1.0, -1.0]. - A present response containing a duplicate ID or an ID outside
0..9is invalid. Return[-1.0, -1.0]for the whole response. - Otherwise, compute metrics from all actual returned files, even when their count differs from
k. Precision is the number of returned dogs divided by the actual returned count. Recall is the number of returned dogs divided by the total number of dogs in the corpus. - Define a metric with denominator zero as
0.0. A present empty response is valid and differs from a missing response.
The received result is supplied as input; no image classifier or external API call is required. Absolute error of at most 10^-9 is accepted.
Function
dogRetrievalMetrics(labels: int[], returnedIds: int[], k: int, responsePresent: boolean) → double[]Examples
Example 1
labels = [1,0,1,0,0,1,0,0,0,1]returnedIds = [0,1,5]k = 3responsePresent = truereturn = [0.6666666666666666,0.5]Two of the three returned files are dogs, and the corpus contains four dogs.
Example 2
labels = [1,0,1,0,0,1,0,0,0,1]returnedIds = [0,1]k = 5responsePresent = truereturn = [0.5,0.25]The API returned only two files despite requesting five. Precision uses the actual two results, while recall still uses all four corpus dogs.
Example 3
labels = [1,0,1,0,0,1,0,0,0,1]returnedIds = []k = 4responsePresent = falsereturn = [-1.0,-1.0]The API response is None, represented by responsePresent=false. Metrics are unavailable rather than zero.
Constraints
labels.length = 10, and each label is0or1.0 <= k <= 10.0 <= returnedIds.length <= 100; each received ID is between-100and100.responsePresentis a boolean.