Deterministic Byte Pair Encoding

For this exercise, assume a corpus of lowercase ASCII words. Train a deterministic word-local byte-pair encoder and return the final tokenization of that same corpus.

1. Initially, each character is one token. Each occurrence in words contributes separately.
2. Count every adjacent pair of current tokens across all words. Overlapping pair occurrences count separately, but word boundaries are never crossed.
3. Select the pair with the largest count. Break a tie by lexicographic order of the left token, then of the right token.
4. In each word, scan left to right and replace every nonoverlapping occurrence of that pair with the concatenation of its two tokens. Recompute frequencies after the complete pass.

Perform at most maxMerges such iterations; stop earlier if no adjacent pair remains. Do not add word-end markers, change character case, remove duplicates or encode another corpus. Return one array of final token strings per input word, preserving word and token order.