Currently I am running the event topic divided into 10 partitions. In all other cases, there is no problem, only snuba-events-subscriptions-consumers are constantly rebalancing or the current offset remains at zero. When I first set it up, I didn’t have this problem. At the time, snuba-events-subscriptions-consumers also worked well. However, once the problem occurs, initializing the offset with “–reset-offsets --to-latest --execute” does not solve the problem. You need a solid solution.
Hi, sorry for the late answer.
I would need some clarifications.
- Does
snuba-transactions-subscriptions-consumershave the same problem? They run with the same consumer, they should behave in the same way. - Do you have some consumer logs of the case where
snuba-events-subscriptions-consumerskeeps rebalancing? How often? Does it actually reshuffle partitions or does it reassign the same to the same consumers? The subscription consumer consumer from two topics:eventsandsnuba-commit-log, so I would be interested in seeing how partitions are assigned there. - Would you be able to start the subscription consumer in debug mode?
-log-level DEBUGand show what is actually doing ?
Thanks
Filippo