Longest Dictionary Tokenization
You are given a text string and a dictionary of token-to-id mappings. Starting from the beginning of text, repeatedly choose the longest dictionary token that matches the current position and output its id. If no dictionary token matches, output the current character itself and advance by one character.
Return the sequence of emitted ids and literal characters.
1Example 1
apple is preferred over app because it is the longest match at index 0.
2Example 2
The first character has no match, then abc is the longest token starting at index 1.
Constraints
Limits and guarantees your solution can rely on.
If multiple dictionary entries have the same token, use the first mapping provided. The tokenization is greedy and scans left to right.