Thanks @ryanbooz. Your assumption is exactly right; I have duplicated the websocket data source x3 to ensure no losses due to outages on its journey across the public internet.
Having spent a few days researching the topic, it seems that de-duping in Kafka is non-trivial. It’s certainly way harder than setting a simple UNIQUE constraint, which just works! In terms of computational complexity, is there any benefit to switching to a INSERT... ON CONFLICT statement instead? From what I understand, both methods can use a btree index on a specific chunk so there shouldn’t be any advantage/benefit in either method. Is this assumption correct?