Unable to solve this Kafka error

ingest-consumer_1 | from cryptography import x509
ingest-consumer_1 | 15:27:16 [INFO] sentry.plugins.github: apps-not-configured
ingest-consumer_1 | Traceback (most recent call last):
ingest-consumer_1 | File “/usr/local/bin/sentry”, line 8, in
ingest-consumer_1 | sys.exit(main())
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/sentry/runner/init.py”, line 166, in main
ingest-consumer_1 | cli(prog_name=get_prog(), obj={}, max_content_width=100)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/click/core.py”, line 722, in call
ingest-consumer_1 | return self.main(*args, **kwargs)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/click/core.py”, line 697, in main
ingest-consumer_1 | rv = self.invoke(ctx)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/click/core.py”, line 1066, in invoke
ingest-consumer_1 | return _process_result(sub_ctx.command.invoke(sub_ctx))
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/click/core.py”, line 1066, in invoke
ingest-consumer_1 | return _process_result(sub_ctx.command.invoke(sub_ctx))
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/click/core.py”, line 895, in invoke
ingest-consumer_1 | return ctx.invoke(self.callback, **ctx.params)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/click/core.py”, line 535, in invoke
ingest-consumer_1 | return callback(*args, **kwargs)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/click/decorators.py”, line 17, in new_func
ingest-consumer_1 | return f(get_current_context(), *args, **kwargs)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/sentry/runner/decorators.py”, line 68, in inner
ingest-consumer_1 | return ctx.invoke(f, *args, **kwargs)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/click/core.py”, line 535, in invoke
ingest-consumer_1 | return callback(*args, **kwargs)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/click/decorators.py”, line 17, in new_func
ingest-consumer_1 | return f(get_current_context(), *args, **kwargs)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/sentry/runner/decorators.py”, line 30, in inner
ingest-consumer_1 | return ctx.invoke(f, *args, **kwargs)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/click/core.py”, line 535, in invoke
ingest-consumer_1 | return callback(*args, **kwargs)
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/sentry/runner/commands/run.py”, line 440, in ingest_consumer
ingest-consumer_1 | get_ingest_consumer(consumer_types=consumer_types, **options).run()
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/sentry/utils/batching_kafka_consumer.py”, line 212, in run
ingest-consumer_1 | self._run_once()
ingest-consumer_1 | File “/usr/local/lib/python2.7/site-packages/sentry/utils/batching_kafka_consumer.py”, line 230, in _run_once
ingest-consumer_1 | raise Exception(msg.error())
ingest-consumer_1 | Exception: KafkaError{code=UNKNOWN_TOPIC_OR_PART,val=3,str=“Subscribed topic not available: ingest-transactions: Broker: Unknown topic or partition”}

Looks like the install script failed to run snuba bootstrap. You may want to run it again.

Getting the same error even after running ./install.sh

Same here. New server with 16 GB RAM / 8 CPU cores. Only Docker is installed, nothing else. It’s a new Sentry installation using the latest sentry/sentry-onpremise but event processing doesn’t work and the ingest-consumer service fails with this error. I removed everything and started again (docker-compose down -t 0 && docker system prune -af --volumes && ./install.sh && docker-compose up -d) but same result.

I tried running docker-compose run --rm snuba-api bootstrap --no-migrate --force and docker-compose run --rm snuba-api migrations migrate --force again but it didn’t help either.

(root@sentry:/var/opt/sentry $) docker-compose run --rm snuba-api bootstrap --no-migrate --force
Starting sentry_onpremise_redis_1      ... done
Starting sentry_onpremise_clickhouse_1 ... done
Starting sentry_onpremise_zookeeper_1  ... done
Starting sentry_onpremise_kafka_1      ... done
+ '[' b = - ']'
+ snuba bootstrap --help
+ set -- snuba bootstrap --no-migrate --force
+ set gosu snuba snuba bootstrap --no-migrate --force
+ exec gosu snuba snuba bootstrap --no-migrate --force
2020-09-22 11:27:42,429 Failed to create topic events
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 94, in bootstrap
    future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str="Topic 'events' already exists."}
2020-09-22 11:27:42,429 Failed to create topic errors-replacements
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 94, in bootstrap
    future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str="Topic 'errors-replacements' already exists."}
2020-09-22 11:27:42,429 Failed to create topic event-replacements
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 94, in bootstrap
    future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str="Topic 'event-replacements' already exists."}
2020-09-22 11:27:42,429 Failed to create topic snuba-commit-log
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 94, in bootstrap
    future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str="Topic 'snuba-commit-log' already exists."}
2020-09-22 11:27:42,430 Failed to create topic cdc
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 94, in bootstrap
    future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str="Topic 'cdc' already exists."}
2020-09-22 11:27:42,430 Failed to create topic outcomes
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 94, in bootstrap
    future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str="Topic 'outcomes' already exists."}
2020-09-22 11:27:42,430 Failed to create topic ingest-sessions
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 94, in bootstrap
    future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
cimpl.KafkaException: KafkaError{code=TOPIC_ALREADY_EXISTS,val=36,str="Topic 'ingest-sessions' already exists."}

(root@sentry:/var/opt/sentry $) docker-compose run --rm snuba-api migrations migrate --force
Starting sentry_onpremise_zookeeper_1 ... done
Starting sentry_onpremise_redis_1      ... done
Starting sentry_onpremise_clickhouse_1 ... done
Starting sentry_onpremise_kafka_1      ... done
+ '[' m = - ']'
+ snuba migrations --help
+ set -- snuba migrations migrate --force
+ set gosu snuba snuba migrations migrate --force
+ exec gosu snuba snuba migrations migrate --force
Finished running migrations

Is fixed now: https://github.com/getsentry/onpremise/issues/683

1 Like