Producer-related configurations
Version: 1.0.0
The producer consists of a pool of buffer space that holds records that haven't yet been transmitted to the server as well as a background I/O thread that is responsible for turning these records into requests and transmitting them to the cluster.
Each cloud connection instantiates a single KafkaProducer
, which is shared among the CloudPublishers.
Attention
The producer implemented in this layer is not transactional nor idempotent.
Acknowledgements
The acks config controls the criteria under which requests are considered complete. The ALL
setting will result in blocking on the full commit of the record, the slowest but most durable setting.
When this property is set to ALL
and the oldest in-flight message is deleted from the data service, the acknowledgement might never be received. This causes the producer to never send new messages. Hence, when using this setting it is recommended tweaking the max.inflight.requests.per.connection
config, the QoS of the produced message (Publisher configurations), or the store settings (DataService layer).
Default configurations
The instantiated KafkaProducer
is initialized with some default configurations that are not definable by the user:
retries=0
: the producer will not try to resend any record whose send fails with a potentially transient error.linger.ms=0
: the producer groups together any records that arrive in between request transmissions into a single batched request. Normally this occurs only under load when records arrive faster than they can be sent out. With this setting to0
, the client will not wait any amount of delay for accumulating larger batches of records. The accumulated records are always sent immediately.request.timeout.ms=1000
: maximum amount of time the client will wait for the response of a request.delivery.timeout.ms=1000
: upper bound on the time to report success or failure after send. This limits the total time that a record will be delayed prior to sending and the time to await acknowledgement from the broker.key.serializer=org.apache.kafka.common.serialization.StringSerializer
: the producer is able to process only records with keys of typejava.lang.String
.value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
: the producer is able to process only records with values of typebyte[]
.
All the other configs are defaulted according to the Kafka Producer configurations documentation.