Connection to Kafka failed When installing

  • ‘[’ b = - ‘]’
  • snuba bootstrap --help
  • set – snuba bootstrap --force
  • set gosu snuba snuba bootstrap --force
  • exec gosu snuba snuba bootstrap --force
    2020-03-15 14:08:34,544 Connection to Kafka failed (attempt 0)
    Traceback (most recent call last):
    File “/usr/src/snuba/snuba/cli/bootstrap.py”, line 55, in bootstrap
    client.list_topics(timeout=1)
    cimpl.KafkaException: KafkaError{code=_TRANSPORT,val=-195,str=“Failed to get metadata: Local: Broker transport failure”}
    2020-03-15 14:08:36,547 Connection to Kafka failed (attempt 1)
    Traceback (most recent call last):
    File “/usr/src/snuba/snuba/cli/bootstrap.py”, line 55, in bootstrap
    client.list_topics(timeout=1)
    cimpl.KafkaException: KafkaError{code=_TRANSPORT,val=-195,str=“Failed to get metadata: Local: Broker transport failure”}
    2020-03-15 14:08:37,630 Failed to create topic events
    Traceback (most recent call last):
    File “/usr/src/snuba/snuba/cli/bootstrap.py”, line 89, in bootstrap
    future.result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 435, in result
    return self.__get_result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 384, in __get_result
    raise self._exception
    cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str=“Topic ‘events’ already exists.”}
    2020-03-15 14:08:37,631 Failed to create topic event-replacements
    Traceback (most recent call last):
    File “/usr/src/snuba/snuba/cli/bootstrap.py”, line 89, in bootstrap
    future.result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 428, in result
    return self.__get_result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 384, in __get_result
    raise self._exception
    cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str=“Topic ‘event-replacements’ already exists.”}
    2020-03-15 14:08:37,631 Failed to create topic snuba-commit-log
    Traceback (most recent call last):
    File “/usr/src/snuba/snuba/cli/bootstrap.py”, line 89, in bootstrap
    future.result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 428, in result
    return self.__get_result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 384, in __get_result
    raise self._exception
    cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str=“Topic ‘snuba-commit-log’ already exists.”}
    2020-03-15 14:08:37,632 Failed to create topic cdc
    Traceback (most recent call last):
    File “/usr/src/snuba/snuba/cli/bootstrap.py”, line 89, in bootstrap
    future.result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 428, in result
    return self.__get_result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 384, in __get_result
    raise self._exception
    cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str=“Topic ‘cdc’ already exists.”}
    2020-03-15 14:08:37,632 Failed to create topic ingest-sessions
    Traceback (most recent call last):
    File “/usr/src/snuba/snuba/cli/bootstrap.py”, line 89, in bootstrap
    future.result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 428, in result
    return self.__get_result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 384, in __get_result
    raise self._exception
    cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str=“Topic ‘ingest-sessions’ already exists.”}
    2020-03-15 14:08:37,632 Failed to create topic errors-replacements
    Traceback (most recent call last):
    File “/usr/src/snuba/snuba/cli/bootstrap.py”, line 89, in bootstrap
    future.result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 428, in result
    return self.__get_result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 384, in __get_result
    raise self._exception
    cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str=“Topic ‘errors-replacements’ already exists.”}
    2020-03-15 14:08:37,632 Failed to create topic outcomes
    Traceback (most recent call last):
    File “/usr/src/snuba/snuba/cli/bootstrap.py”, line 89, in bootstrap
    future.result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 428, in result
    return self.__get_result()
    File “/usr/local/lib/python3.7/concurrent/futures/_base.py”, line 384, in __get_result
    raise self._exception
    cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str=“Topic ‘outcomes’ already exists.”}
    2020-03-15 14:08:37,658 Tables for dataset transactions created.
    2020-03-15 14:08:37,683 Tables for dataset events created.
    2020-03-15 14:08:37,688 Tables for dataset groupedmessage created.
    2020-03-15 14:08:37,695 Tables for dataset sessions created.
    2020-03-15 14:08:37,713 Tables for dataset events_migration created.
    2020-03-15 14:08:37,715 Tables for dataset outcomes_raw created.
    2020-03-15 14:08:37,719 Tables for dataset groupassignee created.
    2020-03-15 14:08:37,719 Tables for dataset discover created.
    2020-03-15 14:08:37,723 Tables for dataset outcomes created.

These seem okay. Those exceptions are printed for information in case something is very off. Looks like the topics were already created so this is not your first run or install, which is fine. I’ll see if we can swallow those specific errors.

Do you have any other issue with your setup?

Thanks.
When the server starts, the following error occurs.
kafka.common.StateChangeFailedException: Failed to elect leader for partition __consumer_offsets-22 under strategy OfflinePartitionLeaderElectionStrategy
kafka_1 | at kafka.controller.PartitionStateMachine$$anonfun$doElectLeaderForPartitions$3.apply(PartitionStateMachine.scala:366)
kafka_1 | at kafka.controller.PartitionStateMachine$$anonfun$doElectLeaderForPartitions$3.apply(PartitionStateMachine.scala:364)
kafka_1 | at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
kafka_1 | at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
kafka_1 | at kafka.controller.PartitionStateMachine.doElectLeaderForPartitions(PartitionStateMachine.scala:364)
kafka_1 | at kafka.controller.PartitionStateMachine.electLeaderForPartitions(PartitionStateMachine.scala:292)
kafka_1 | at kafka.controller.PartitionStateMachine.doHandleStateChanges(PartitionStateMachine.scala:210)
kafka_1 | at kafka.controller.PartitionStateMachine.handleStateChanges(PartitionStateMachine.scala:133)
kafka_1 | at kafka.controller.PartitionStateMachine.triggerOnlinePartitionStateChange(PartitionStateMachine.scala:123)
kafka_1 | at kafka.controller.PartitionStateMachine.triggerOnlinePartitionStateChange(PartitionStateMachine.scala:109)
kafka_1 | at kafka.controller.PartitionStateMachine.startup(PartitionStateMachine.scala:66)
kafka_1 | at kafka.controller.KafkaController.onControllerFailover(KafkaController.scala:260)
kafka_1 | at kafka.controller.KafkaController.kafka$controller$KafkaController$$elect(KafkaController.scala:1221)
kafka_1 | at kafka.controller.KafkaController$Startup$.process(KafkaController.scala:1134)
kafka_1 | at kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply$mcV$sp(ControllerEventManager.scala:89)
kafka_1 | at kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:89)
kafka_1 | at kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:89)
kafka_1 | at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:31)
kafka_1 | at kafka.controller.ControllerEventManager$ControllerEventThread.doWork(ControllerEventManager.scala:88)
kafka_1 | at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
kafka_1 | [2020-03-19 09:04:43,651] ERROR [Controller id=1005 epoch=41] Controller 1005 epoch 41 failed to change state for partition __consumer_offsets-30 from OfflinePartition to OnlinePartition (state.change.logger)
kafka_1 | kafka.common.StateChangeFailedException: Failed to elect leader for partition __consumer_offsets-30 under strategy OfflinePartitionLeaderElectionStrategy
kafka_1 | at kafka.controller.PartitionStateMachine$$anonfun$doElectLeaderForPartitions$3.apply(PartitionStateMachine.scala:366)
kafka_1 | at kafka.controller.PartitionStateMachine$$anonfun$doElectLeaderForPartitions$3.apply(PartitionStateMachine.scala:364)
kafka_1 | at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
kafka_1 | at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
kafka_1 | at kafka.controller.PartitionStateMachine.doElectLeaderForPartitions(PartitionStateMachine.scala:364)
kafka_1 | at kafka.controller.PartitionStateMachine.electLeaderForPartitions(PartitionStateMachine.scala:292)
kafka_1 | at kafka.controller.PartitionStateMachine.doHandleStateChanges(PartitionStateMachine.scala:210)
kafka_1 | at kafka.controller.PartitionStateMachine.handleStateChanges(PartitionStateMachine.scala:133)
kafka_1 | at kafka.controller.PartitionStateMachine.triggerOnlinePartitionStateChange(PartitionStateMachine.scala:123)
kafka_1 | at kafka.controller.PartitionStateMachine.triggerOnlinePartitionStateChange(PartitionStateMachine.scala:109)
kafka_1 | at kafka.controller.PartitionStateMachine.startup(PartitionStateMachine.scala:66)
kafka_1 | at kafka.controller.KafkaController.onControllerFailover(KafkaController.scala:260)
kafka_1 | at kafka.controller.KafkaController.kafka$controller$KafkaController$$elect(KafkaController.scala:1221)
kafka_1 | at kafka.controller.KafkaController$Startup$.process(KafkaController.scala:1134)
kafka_1 | at kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply$mcV$sp(ControllerEventManager.scala:89)
kafka_1 | at kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:89)
kafka_1 | at kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:89)
kafka_1 | at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:31)
kafka_1 | at kafka.controller.ControllerEventManager$ControllerEventThread.doWork(ControllerEventManager.scala:88)
kafka_1 | at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
kafka_1 | [2020-03-19 09:04:43,654] ERROR [Controller id=1005 epoch=41] Controller 1005 epoch 41 failed to change state for partition __consumer_offsets-8 from OfflinePartition to OnlinePartition (state.change.logger)
kafka_1 | kafka.common.StateChangeFailedException: Failed to elect leader for partition __consumer_offsets-8 under strategy OfflinePartitionLeaderElectionStrategy
kafka_1 | at kafka.controller.PartitionStateMachine$$anonfun$doElectLeaderForPartitions$3.apply(PartitionStateMachine.scala:366)
kafka_1 | at kafka.controller.PartitionStateMachine$$anonfun$doElectLeaderForPartitions$3.apply(PartitionStateMachine.scala:364)
kafka_1 | at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
kafka_1 | at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
kafka_1 | at kafka.controller.PartitionStateMachine.doElectLeaderForPartitions(PartitionStateMachine.scala:364)

Another issue is that my project can’t collect issues. Why?

All I can say is your Kafka instance has some issues but I don’t know what. It can be a corrupt volume so if you don’t have any data to lose, you may try re-creating the sentry-kafka volume. This is also the reason why you cannot get any events.

Okay! I try it.