On MQTT 5, an mqtt5 object reports the publish properties — userProperties, contentType, correlationData (base64), responseTopic, payloadFormatIndicator and messageExpiryInterval. It is omitted on MQTT 3.1.1, which has no such properties, and omitted when the publisher set none, so its presence means something.

userProperties is an array of {name, value} pairs, not an object, because MQTT 5 permits a repeated name and requires the order preserved. Measured against Mosquitto: publishing dup=first, other=in-between, dup=second delivers all three, in that order — a JSON object would have kept one dup and lost the ordering.

An MQTT payload is arbitrary bytes — protobuf, CBOR, an image, anything compressed — and JSON cannot carry bytes. So the envelope sends payload as text when the bytes are valid UTF-8 and as base64 when they are not, and payloadEncoding is always present, "text" or "base64", because a base64 string is indistinguishable by inspection from a text payload that happens to look like base64.

The raw format (payload-envelope: false) has nowhere to put that label, and SSE is a UTF-8 text protocol whose data: lines cannot carry arbitrary bytes at all. A non-text payload is therefore sent as unlabelled base64 and the service warns once.

A consumer that wants live data must ask both. They only exist with payload-envelope: true; the raw format has nowhere to put them.

retain is not the publisher’s retain flag. MQTT 3.1.1 §3.3.1.3 has the server set RETAIN on delivery only when the message is sent as the result of a new subscription, and clear it for an established subscription however the publisher set it. So retain: true means "I was given this because I had just subscribed", not "the publisher asked for this to be retained" — which a subscriber cannot know. Measured both ways with mosquitto_sub -F '%r': the same retained publish arrives with the flag set to a fresh subscription and clear to an established one.

They are orthogonal, and the combination that catches people is replay: false, retain: true: the first client to subscribe to a filter receives the broker’s retained value through the live path, so it arrives looking like an event that has just happened. A genuinely new event is neither replayed nor retained.

It is how a departed client is noticed at all. Nothing reads an SSE connection after the handshake, so the only way the server learns a client is gone is a write to its socket failing. On a busy topic that happens on the next message; on a quiet one it may never be attempted.

Until it is, the connection’s router listener stays registered, filling a queue nobody drains, and its broker subscription stays in place — so every message matching both that filter and a broader one is delivered to the module twice, duplicating SSE events for other clients, duplicating custom-plugin callbacks, and duplicating MongoDB documents under id-strategy: auto. With clean-session: false the leaked broker subscription outlives the process, because it lives in the broker’s session.

The periodic comment turns that into a bounded wait. Detection takes up to two periods, not one: the first write into a half-closed socket succeeds, and only the next one fails.

capacity (10000) is a number of messages and max-bytes (64 MB) is how much heap they may take; whichever is reached first applies backpressure. Both are needed: ten thousand readings of a hundred bytes are a megabyte, ten thousand images of a hundred kilobytes are a gigabyte. A message’s weight is its payload plus its topic plus a fixed allowance for the objects around them.

buffer.strategy, default blocking-queue. buffer.max-wait-ms bounds how long blocking-queue waits for room; 0 — the default — waits indefinitely.

An unrecognised strategy fails at startup rather than silently falling back.

id-strategy, default auto.

Two values, not three, and the reason is worth stating because it also answers every future request for a clever third: nothing the receiver computes locally can be stable across two receptions of the same message. A redelivery is a fresh reception, so receivedAt differs, and the retain flag depends on subscription timing. Convergence needs an identity that travels with the message, which means the publisher has to supply one. That is a boundary of MQTT 3.1.1, not a gap in this module.

A duplicate-key error is treated as "already stored" under either strategy, so neither retries nor dead-letters a document that is in the database. That matters for auto too: when a write fails at the connection level the whole batch is re-sent, and part of it may already have landed.

id-field (default messageId) must be set and non-blank when id-strategy is payload-field. Both keys are validated at startup, because a typo would otherwise disable deduplication silently.

payload-field only applies to text payloads; a binary one falls through to auto, since there is no JSON document to read a field from. And if the payload is not valid JSON or the field is absent, the failure is swallowed: no _id is computed, and that one document falls back to a plain insert instead of an upsert — deduplication is lost silently, per message, rather than failing the write.

Only payload-field converges, and it converges everywhere: across a redelivery to the same instance, and across nodes that all receive the same broker message. auto converges nowhere — every redelivery is a new document, which is a legitimate choice when you would rather keep every delivery than key on something the publisher controls.

Documents use BSON types rather than strings, so the collection can be queried and indexed as what it is:

receivedAtNanos exists because BSON dates are milliseconds and receivedAt carries nanoseconds. Two messages inside one millisecond are ordinary at sensor rates, and a collection meant to be replayable must not lose the order they arrived in.

Every document also records retain, so the collection records the event rather than an interpretation of it. Without it, a value delivered as a topic’s stored last-known-state — possibly days old, and possibly already in the collection from before — is indistinguishable from a measurement just taken. In a steady-state deployment it is false on nearly every document, and true mainly on the messages delivered just after a (re)start, which is exactly when a row is most likely to be a re-record of an old value.

Two things a subscriber cannot know, so the collection cannot record them. Both limit how faithfully a stored stream can be replayed, and neither is a gap in this module:

Everything else a replay needs is there: topic, payload byte for byte, QoS, ordering to the nanosecond, and the MQTT 5 properties.

dead-letter-collection, default mqtt-dead-letter, in the same database as the sink whose write failed.

Only documents MongoDB refuses — a failed validation, a size or type it will not take — end up there, after drain.max-retries attempts. Each keeps the document as it was plus a _deadLetter sub-document naming the collection it was meant for, the reason, and when. Being documents in a collection, they can be queried, repaired and re-inserted with the same REST API as anything else.

max-retries governs only this path: a MongoDB that is unreachable is retried indefinitely instead, because the messages have already been acknowledged to the broker and there is nowhere else for them to be. A MongoDB that is down therefore produces no dead letters — a dead-letter queue in a stopped server would be no use anyway.

If the dead-letter write fails too — the collection validates its documents as well, say — it is logged and those documents are lost; nothing is propagated into the drain loop. Re-ingesting them through an API of their own is tracked in #607.

Every mongo-sink entry must carry all three of topic, database and collection, each a string; a missing or mistyped key fails at startup naming the entry. An absent or empty mongo-sink list is legal and simply means nothing is persisted.