Multiple fafka events partitions vs multiple post-process-forwarders

Say I want to increase throughput of my snuba/kafka chain and I’ve added more partitions to events topic and processes to snuba (as described here - How can I scale out the snuba?! - #4 by seungjinlee ) - because the event flow is frequently stopped and I need to flush the kafka topics to resume it. My question is - multiple post-process-forwarders are now mandatorial or are they optional ? In both cases, do I merey run more of them, or do I need to add some runtime switches to organize them in any way ?


If you have ingestion issues, adding dedicated worker services should be the first step, not scaling Snuba out:

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.