KeyError while running snuba-subscription-consumer-events

1

Hi Sentry Team.

We were missing running two sentry and snuba components (subscription-consumer-events and subscription-consumer-transactions) and today when we started them we got the following stacktrace.

Traceback (most recent call last):
  File "/usr/local/bin/snuba", line 33, in <module>
    sys.exit(load_entry_point('snuba', 'console_scripts', 'snuba')())
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/src/snuba/snuba/cli/subscriptions.py", line 232, in subscriptions
    batching_consumer.run()
  File "/usr/local/lib/python3.8/site-packages/arroyo/processing/processor.py", line 108, in run
    self._run_once()
  File "/usr/local/lib/python3.8/site-packages/arroyo/processing/processor.py", line 146, in _run_once
    self.__processing_strategy.submit(self.__message)
  File "/usr/local/lib/python3.8/site-packages/arroyo/processing/strategies/batching.py", line 150, in submit
    result = self.__worker.process_message(message)
  File "/usr/src/snuba/snuba/subscriptions/worker.py", line 262, in process_message
    for task in self.__schedulers[message.partition.index].find(tick.timestamps)
KeyError: 1

We have 8 partitions for the topic. I came across similar thread with no answer Snuba-events-subscriptions-consumers is not working

Thank you for your help.

Regards,
Pradeep

2

Found the issue.

If you using more partitions than 1, you would need to pass --partitions <num-of-partitions> flag to both snuba-subscription-consumer-events and snuba-subscription-consumer-transactions consumers otherwise they will not work and your alerting on metrics/transactions won’t work.