We are continuously testing the part where there is a problem related to the topic

On-Premise
1

Testing continues following the link below.

We run everything for the first time, and we are testing in the default state with 1 partition per topic.

snuba-subscription-consumer-events snuba-subscription-consumer-transactions subscription-consumer-events subscription-consumer-transactions
  1. snuba-subscription-consumer-events was processed, but with the following error:

2021-09-15 10:58:30,137 Flushing batch via worker
2021-09-15 10:58:30,138 Worker flush took 0ms
2021-09-15 10:58:30,138 Committing offsets for batch
2021-09-15 10:58:30,143 Waited 0.0046 seconds for offsets to be committed to <snuba.subscriptions.consumer.TickConsumer object at 0x7f23fb54ac40>.
2021-09-15 10:58:30,143 Committed offsets: {Partition(topic=Topic(name=‘events’), index=0): 54935}
2021-09-15 10:58:30,143 Offset commit took 4ms
2021-09-15 10:58:33,100 Could not construct valid time interval between MessageDetails(offset=54950, timestamp=datetime.datetime(2021, 9, 15, 10, 58, 30, 71000)) and Message(partition=Partition(topic=Topic(name=‘events’), index=0), offset=54951)!
Traceback (most recent call last):
File “/usr/src/snuba/snuba/subscriptions/consumer.py”, line 129, in poll
time_interval = Interval(previous_message.timestamp, message.timestamp)
File “”, line 5, in init
File “/usr/src/snuba/snuba/utils/types.py”, line 67, in post_init
raise InvalidRangeError(self.lower, self.upper)

=> We solved this problem by setting KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: LogAppendTime in the kafka docker-compose settings. No more error logs

  1. snuba-subscription-consumer-transactions did not work itself. We were using snuba 20.11.0 version, but looking closely, there was a problem with the setting of 20.11.0. I set --commit-log-group to snuba-transactions-consumers in docker-compose configuration of snuba-subscription-consumer-transactions, but found that there is no such group. When I looked at the snuba version of 20.11.1, I saw that it was changed to transactions_group and modified the image and docker-compose command. But it doesn’t work. No error log occurred
  • set – snuba subscriptions --auto-offset-reset=latest --consumer-group=snuba-transactions-subscriptions-consumers --log-level=debug --topic=events --result-topic=transactions-subscription-results --dataset=transactions --commit-log-topic=snuba-commit-log --commit-log-group=transactions_group --delay-seconds=60 --schedule-ttl=60
  • set gosu snuba snuba subscriptions --auto-offset-reset=latest --consumer-group=snuba-transactions-subscriptions-consumers --log-level=debug --topic=events --result-topic=transactions-subscription-results --dataset=transactions --commit-log-topic=snuba-commit-log --commit-log-group=transactions_group --delay-seconds=60 --schedule-ttl=60
  • exec gosu snuba snuba subscriptions --auto-offset-reset=latest --consumer-group=snuba-transactions-subscriptions-consumers --log-level=debug --topic=events --result-topic=transactions-subscription-results --dataset=transactions --commit-log-topic=snuba-commit-log --commit-log-group=transactions_group --delay-seconds=60 --schedule-ttl=60
    2021-09-15 11:56:04,805 Waiting for commit log consumer to receieve assignment…
    2021-09-15 11:56:08,003 Commit log consumer received assignment: {Partition(topic=Topic(name=‘snuba-commit-log’), index=0): 0}
    2021-09-15 11:56:08,003 Commit log consumer has started.
    2021-09-15 11:56:08,005 Starting <concurrent.futures.thread.ThreadPoolExecutor object at 0x7febedc7d8b0> with 8 workers…
    2021-09-15 11:56:08,005 Starting
    2021-09-15 11:56:11,048 New partitions assigned: {Partition(topic=Topic(name=‘events’), index=0): 0}
    2021-09-15 11:56:11,048 Initialized processing strategy: <snuba.utils.streams.processing.strategies.batching.BatchProcessingStrategy object at 0x7febedc7dfd0>
  1. The “subscription-consumer-events” and “subscription-consumer-transactions” groups do not work properly. They are looking at the “events-subscription-results” and “transactions-subscription-results” topics, respectively, which are created by the two preceding “snuba-subscription-consumer-events” and “snuba-subscription-consumer-transactions” groups, respectively. Even though it is created, an error log occurs and it does not run properly.

11:56:06 [INFO] sentry.plugins.github: apps-not-configured
11:56:06 [WARNING] batching-kafka-consumer: Topic ‘events-subscription-results’ or its partitions are not ready, retrying…
11:56:12 [INFO] sentry.snuba.query_subscription_consumer: query-subscription-consumer.on_assign (partitions=u’[TopicPartition{topic=events-subscription-results,partition=0,offset=-1001,error=None}]’ offsets=u’{0: None}')

Currently, we are operating the sentry service based on version 20.11.0.
In the case of clickhouse, it has been customized as a distributed table for cluster configuration.
How can I fix this?

2

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.