Kafka Topics Configuration

As a followup to this topic discussed:Snuba consumer : Exception: Broker: Not enough in-sync replicas I would like to fine-tune the topic configurations based on my need. “ReplicationFactor” is one among them.

AWS - MSK Configuration:
auto.create.topics.enable = true
delete.topic.enable = true
default.replication.factor = 3
min.insync.replicas = 2

Eventhough I have set the “default.replication.factor” as 3, the below topics are created with ReplicationFactor: 1.

  • errors-replacements
  • event-replacements
  • events
  • ingest-sessions
  • outcomes
  • snuba-commit-log

Example describe topic command (masked zookeeper connection string):

# ./kafka-topics.sh --describe --topic errors-replacements --zookeeper xyz.cpm:2181

Topic: errors-replacements	PartitionCount: 1	ReplicationFactor: 1	Configs:
Topic: errors-replacements	Partition: 0	Leader: 3	Replicas: 3	Isr: 3

While other topics(like ingest-attachments, ingest-events, ingest-transactions) are created with the default replication factor.

Example describe topic command (masked zookeeper connection string):

# ./kafka-topics.sh --describe --topic ingest-transactions --zookeeper xyz.cpm:2181

Topic: ingest-transactions	PartitionCount: 1	ReplicationFactor: 3	Configs:
Topic: ingest-transactions	Partition: 0	Leader: 2	Replicas: 2,3,1	Isr: 2,3,1

Can someone please point me where the six topics (errors-replacements, event-replacements, events, ingest-sessions, outcomes, snuba-commit-log) mentioned above can be configured based on my requirement.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.