As a followup to this topic discussed:Snuba consumer : Exception: Broker: Not enough in-sync replicas I would like to fine-tune the topic configurations based on my need. “ReplicationFactor” is one among them.
AWS - MSK Configuration:
auto.create.topics.enable = true
delete.topic.enable = true
default.replication.factor = 3
min.insync.replicas = 2
Eventhough I have set the “default.replication.factor” as 3, the below topics are created with ReplicationFactor: 1.
- errors-replacements
- event-replacements
- events
- ingest-sessions
- outcomes
- snuba-commit-log
Example describe topic command (masked zookeeper connection string):
# ./kafka-topics.sh --describe --topic errors-replacements --zookeeper xyz.cpm:2181
Topic: errors-replacements PartitionCount: 1 ReplicationFactor: 1 Configs:
Topic: errors-replacements Partition: 0 Leader: 3 Replicas: 3 Isr: 3
While other topics(like ingest-attachments, ingest-events, ingest-transactions) are created with the default replication factor.
Example describe topic command (masked zookeeper connection string):
# ./kafka-topics.sh --describe --topic ingest-transactions --zookeeper xyz.cpm:2181
Topic: ingest-transactions PartitionCount: 1 ReplicationFactor: 3 Configs:
Topic: ingest-transactions Partition: 0 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1
Can someone please point me where the six topics (errors-replacements, event-replacements, events, ingest-sessions, outcomes, snuba-commit-log) mentioned above can be configured based on my requirement.