Skip to content

Cloud Publisher Configuration

Version: 1.0.0

The com.eurotech.framework.kafka.client.cloudconnection.publisher.KafkaClientPublisher allows interaction with the KafkaProducer instance of the Cloud Connection. The configuration properties for the publisher are detailed below.

Events created by the underlying KafkaProducer are sent as ProducerRecord. Such records have a fixed type for the key and the value and are serialized as follows:

Producer topic

This configuration is mandatory. It sets the Kafka topic to which send the created events. Valid characters for Kafka topics match the following pattern:

[a-zA-Z0-9\._\-]
However, one thing to keep in mind is that due to limitations in metric names, topics with a period (.) or underscore (_) could collide. To avoid issues it is best to use either, but not both.

Key

This configuration is optional. When specified, all the messages of this publisher are sent to a single partition. In Kafka, the messages are guaranteed to be processed in order only if they share the same key, so specify the key only if the application requires message ordering.

The default partitioner used by the producer in Kafka works as follows. The producer first calculates a numeric hash of the key using murmur2 algorithm, and then selects the partition number by the following formula:

murmur2(key) % number_of_partitions
Where % is the modulus operation.

For example, the murmur2 of the string testkey is 2871421366. If the topic has 10 partitions, 2871421366 % 10 = 6. Hence, using the default partitioner, every message send with key testkey will land in partition number 6.

Publish metrics & Publish position

When set to false, the publisher will strip off such data from the KuraPayload before sending it. Metrics and position data are always ignored when the Payload Encoding property of the KafkaClientCloudEndpoint section is set to RAW.

Priority

Used by the data service, sets the priority of the message. This priority is used to determine the republishing order of the unpublished messages. A message reordering may occur.

Quality of service (QoS)

This parameter is used by the data service to guarantee that the message is delivered at most once (QoS = 0) or at least once (QoS > 0).