Document Enrichment Stream Aggregator
Problem statement
Aggregate semantic enrichment events by document and publish each document at most once.
requiredEnrichers lists unique enricher IDs in their configured order. Process each row in operations, whose timestamps are nondecreasing:
["event", timestamp, documentId, enricherId, enrichment]consumes an enrichment event.["advance", timestamp]advances time without consuming an event.
Before processing an operation at time t, publish every pending document whose deadline is at most t. A document's deadline is ten seconds after its first event from a required enricher. Order simultaneous expirations by ascending deadline and then lexicographically by document ID.
Ignore events from unconfigured enrichers, and do not start a timer for them. For a required enricher, store its latest value. Publish a document immediately after an event completes all required enrichers. Once a document is published, ignore all later events for it.
Return publications in order. Each publication row begins with the document ID, followed by enricherId and enrichment pairs for the required enrichers that are present, in configured order. Omit missing enrichers from an incomplete timeout publication.
Function
aggregateEnrichments(requiredEnrichers: String[], operations: String[][]) → String[][]Examples
Example 1
requiredEnrichers = ["Sentiment Analysis","Entity Linking"]operations = [["event","0","doc1","Sentiment Analysis","positive"],["event","1","doc2","Entity Linking","[\"AAPL\"]"],["event","2","doc1","Topic Classification","[\"AUTO\"]"],["event","3","doc1","Entity Linking","[\"TSLA\"]"],["advance","11"]]return = [["doc1","Sentiment Analysis","positive","Entity Linking","[\"TSLA\"]"],["doc2","Entity Linking","[\"AAPL\"]"]]doc1 publishes immediately at time 3 with both required enrichments. The topic event is ignored. At time 11, doc2 reaches its deadline and publishes with only Entity Linking.
Example 2
requiredEnrichers = ["A","B"]operations = [["event","0","docB","A","x"],["event","0","docA","A","y"],["event","10","docC","A","z"],["advance","20"]]return = [["docA","A","y"],["docB","A","x"],["docC","A","z"]]docA and docB expire before the event at time 10 is consumed, and their equal deadlines are broken by document ID. docC then expires at time 20.
Example 3
requiredEnrichers = ["A","B"]operations = [["event","0","doc","A","old"],["event","1","doc","A","new"],["event","2","doc","B","done"]]return = [["doc","A","new","B","done"]]The second A event replaces the first value. The B event completes the document, which publishes immediately with the latest configured values.
Constraints
1 <= requiredEnrichers.length <= 20.- Required enricher IDs are unique.
1 <= operations.length <= 100000.- Operation timestamps are nondecreasing integers from
0through1000000000. - Document IDs, enricher IDs, and enrichment strings contain between
1and100visible ASCII characters. - Every operation has exactly the fields required by its type.