FastPrepReliable LLM Call with Independent Retry Budgets
Problems/Reliable LLM Call with Independent Retry Budgets

Reliable LLM Call with Independent Retry Budgets

Stripe logoStripe● MediumNEW GRADOA

Updated Nov 2026

QuestionFunctionExamplesConstraints

Problem statement

Source note: Reported as the second question of a Stripe new-grad online assessment: write a helper that calls an LLM client, checks each answer with a validator, and retries client errors and validation failures on two separate budgets. A failed answer goes back into the conversation with a correction, the caller's list stays untouched, and running out of retries raises an error that carries the call count, the last answer and every failed answer. Budgets can be zero.

The report leaves several details open. Here, the client and validator are replaced by a scripted list of outcomes, the client-error message and the reason used when the validator just says no are fixed strings we chose, running out of validation retries reports validation_error with a matching message, and the result and the history of the last call are returned as a list of strings.

Our version is about 70% the same as the reported question.

You are writing reliable_call, a helper that sends a conversation to an LLM API, checks each response with a validator, and retries when something goes wrong. In this exercise there is no real API. Instead, a script says what each call does. Calls are counted from 1: call 1 has the outcome outcomes[0], the response text responses[0] and the reason reasons[0], call 2 uses index 1, and so on. In the rules below, i is the list index for the current call (call i + 1).

Each message is a string role: content. The history starts as a copy of messages; messages itself is never changed. Every API call is sent the current history. The outcomes are:

  • "CLIENT_ERROR": the API call fails with a client error and returns no response. If the client retries used so far equal maxClientRetries, stop with client_error. Otherwise use one client retry, add CLIENT_ERROR_MESSAGE, which is "user: The API call failed. Please try again.", to the history, and call again.
  • "VALID": the call returns responses[i] and the validator accepts it. Stop with success.
  • "INVALID": the call returns responses[i] and the validator returns false. The reason is validation returned False.
  • "VALIDATION_ERROR": the call returns responses[i] and the validator raises a validation error whose reason is reasons[i].

After "INVALID" or "VALIDATION_ERROR", the response is a failed response. If the validation retries used so far equal maxValidationRetries, stop with validation_error. Otherwise use one validation retry, add "assistant: " + response to the history, then add "user: That response was invalid: " + reason + ". Please try again with valid JSON.", and call again.

The two budgets are independent. A client error uses only a client retry, a failed response uses only a validation retry, and neither count is ever reset. A budget of 0 allows no retry for that kind of failure.

Return a list of strings, in this order:

  1. The status: "success", "client_error" or "validation_error". For an error, this is its failure_reason.
  2. For a success, the accepted response. For an error, its message: "Client error limit reached" or "Validation error limit reached".
  3. attempts: the total number of API calls made, as a decimal string.
  4. last_response: the response of the most recent call that returned one, or "None" if every call failed with a client error.
  5. The number of failed responses as a decimal string, followed by the failed responses in call order (all_responses).
  6. The history that was sent with the last API call, one message per string.

Function

reliableCall(messages: String[], outcomes: String[], responses: String[], reasons: String[], maxClientRetries: int, maxValidationRetries: int) → String[]

Examples

Example 1

messages = ["system: Reply with JSON only.","user: What is the total of order 17?"]outcomes = ["CLIENT_ERROR","INVALID","VALID"]responses = ["","Sure! It is 42.","{\"total\": 42}"]reasons = ["","",""]maxClientRetries = 1maxValidationRetries = 1return = ["success","{\"total\": 42}","3","{\"total\": 42}","1","Sure! It is 42.","system: Reply with JSON only.","user: What is the total of order 17?","user: The API call failed. Please try again.","assistant: Sure! It is 42.","user: That response was invalid: validation returned False. Please try again with valid JSON."]
  • Call 1 fails with a client error. One client retry is allowed, so the client error message is added and the call is retried.
  • Call 2 returns Sure! It is 42., and the validator returns false. One validation retry is allowed, so the response is added as an assistant message, followed by the correction with the reason validation returned False.
  • Call 3 returns {"total": 42}, which passes. The result is a success after 3 calls, with one failed response. The last five strings are the messages sent with call 3.

Example 2

messages = ["system: Reply with JSON only.","user: What is the total of order 17?"]outcomes = ["VALIDATION_ERROR","VALID"]responses = ["{total: 42}","{\"total\": 42}"]reasons = ["missing quotes around total",""]maxClientRetries = 2maxValidationRetries = 0return = ["validation_error","Validation error limit reached","1","{total: 42}","1","{total: 42}","system: Reply with JSON only.","user: What is the total of order 17?"]
  • Call 1 returns {total: 42}, and the validator raises a validation error.
  • The validation budget is 0, so there is no retry: the call stops with validation_error after 1 call. The second scripted outcome is never used.
  • The only call was sent the two original messages.

Example 3

messages = ["system: Reply with JSON only.","user: What is the total of order 17?"]outcomes = ["VALIDATION_ERROR","CLIENT_ERROR","CLIENT_ERROR","CLIENT_ERROR"]responses = ["[]","","",""]reasons = ["expected an object","","",""]maxClientRetries = 2maxValidationRetries = 1return = ["client_error","Client error limit reached","4","[]","1","[]","system: Reply with JSON only.","user: What is the total of order 17?","assistant: []","user: That response was invalid: expected an object. Please try again with valid JSON.","user: The API call failed. Please try again.","user: The API call failed. Please try again."]
  • Call 1 returns [] and fails validation, which uses the one validation retry.
  • Calls 2 and 3 fail with client errors and use both client retries. The validation failure did not use any of them, because the budgets are separate.
  • Call 4 fails with a client error and no client retry is left, so the result is client_error after 4 calls. The last response received was [], from call 1.

Constraints

  • 1 <= messages.length <= 10
  • Each message has the form role: content, where role is system, user or assistant and content has 1 to 40 characters.
  • 1 <= outcomes.length <= 50
  • responses.length == reasons.length == outcomes.length
  • outcomes[i] is "CLIENT_ERROR", "VALID", "INVALID" or "VALIDATION_ERROR".
  • responses[i] is "" when outcomes[i] is "CLIENT_ERROR". Otherwise it has 1 to 40 characters and is not "None".
  • reasons[i] is "" unless outcomes[i] is "VALIDATION_ERROR". Then it has 1 to 40 characters and does not end with a period.
  • Message contents, responses and reasons use only letters, digits, spaces and the symbols { } [ ] " : , . ! ? ' - _, with no leading or trailing space and no two spaces in a row.
  • 0 <= maxClientRetries, maxValidationRetries <= 20
  • The script is long enough: the process always stops at or before the last outcome. Outcomes after the stopping call are never used.

More Stripe Online Assessment questions

  • Asynchronous Event ProcessingOA · Seen Nov 2026
  • Maximum Game Score with Fixed Forward JumpsOA · Seen Nov 2026
  • Request Routing SystemOA · Seen Nov 2026
  • Financial Account LedgerOA · PHONE SCREEN · Seen Nov 2026
  • Chat Billing CalculationOA · Seen Nov 2026
  • Deployment Window SchedulerOA · Seen Nov 2026
  • Subscription and Usage-Based Billing CalculatorOA · Seen Nov 2026
  • Split Tender Part 4 - RefundsOA · PHONE SCREEN · Seen Nov 2026
  • Incident MonitorOA · Seen Nov 2026
  • Split Tender Part 3 - SurchargesOA · Seen Nov 2026

Similar questions at other companies

  • Parcel Event TrackingAirbnb · OAAirbnb interview questions
  • Communications HandlerAkuna Capital · OAAkuna Capital interview questions
  • Text Editor OperationsMicrosoft · OAMicrosoft interview questions
  • Versioned Recipe SystemThe Trade Desk · OA
  • In-Memory Filesystem CommandsPerplexity AI · PHONE SCREEN · ONSITE INTERVIEWPerplexity AI interview questions
  • Stateful Log Transform and SearchRippling · PHONE SCREEN · ONSITE INTERVIEWRippling interview questions
  • In-Memory Vault File SystemHarvey · ONSITE INTERVIEWHarvey interview questions
  • File Storage ServiceCresta · PHONE SCREENCresta interview questions
  • Query JSON Records with WHERE and ORDER BYGitHub · ONSITE INTERVIEW
  • Return Every Matching DecryptionDuolingo · PHONE SCREENDuolingo interview questions

Interview reports that mention this question

  • Stripe New Grad SWE OA and Phone Screen: 2 Experiences

Stripe interview experiences

  • Stripe Experienced SWE OA and Onsite Questions: 3 Experiences
  • Stripe SWE Intern Linked Merchants Team Screen: 1 Experience
  • Stripe OA: What It Tests (2026)
  • Stripe SWE Onsite over Two Days: 1 Experience
  • Stripe New Grad SWE Phone Screen and Onsite: 2 Experiences
All Stripe interview questionsSee Stripe hiring insights
Saved
class Solution {
    public String[] reliableCall(String[] messages, String[] outcomes, String[] responses, String[] reasons, int maxClientRetries, int maxValidationRetries) {
        // Write your code here.
    }
}
Checking account…