Sentry consuming all CPU

1

Hi I have been working with Sentry for a month so, I configured the SMTP, and now I’m facing an issue with our on premise version

Main issue is that workers seems not to be enough to handle the requirements, but main thing that changed was the SES configuration
[ERROR] sentry.errors.events: preprocess.failed.empty (cache_key=u
I read the thread regarding to purge the queue Worker failing with `sentry.errors.events: preprocess.failed.empty`
I have flushed the Redis memory, and still having a peak period of more than 90% of CPU.
For SES configuration, I configured it .env file and starting up with it. So I’m not receiving more errors regarding timeouts.

The queues are on this way when it gets blocked
activity.notify 0
alerts 0
assemble 0
auth 0
buffers.process_pending 0
cleanup 0
commits 0
counters-0 8955
default 0
digests.delivery 0
digests.scheduling 0
email 0
events.index_event_tags 0
events.preprocess_event 424227
events.process_event 0
events.reprocess_events 0
events.reprocessing.preprocess_event 0
events.reprocessing.process_event 0
events.save_event 0
integrations 0
merge 0
options 0
reports.deliver 0
reports.prepare 0
search 0
stats 0
triggers-0 0
unmerge 0
update 0

I will appreciate any help.

2

Yeah, I mean, this doesn’t really surprise me at all. You are encouraged, and likely need, to run more workers. They’re are all stateless and horizontally scalable.

3

Thanks a lot for your quick response. I changed the configuration to test the double of the ones I had ,not it is 16 workers, as per your documentation max is 36 https://docs.sentry.io/server/performance/ What happend is that it doesn’t matter how many workers it has (16,10,5) , at one time it start sending the errors (it takes minutes to reach 99% of CPU usage):

23:14:59 [ERROR] sentry.errors.events: preprocess.failed.empty (cache_key=u’:3’)
23:14:59 [ERROR] sentry.errors.events: preprocess.failed.empty (cache_key=u’:3’)

I have tried configure more workers and it didn’t work my sentry instance got freezed.

The machine where it runs has 8 CPU and 32 GiB of RAM, RAM isn’t taking not even 20% but CPU gots crazy.

Another thing I tried was customize the inbound https://docs.sentry.io/accounts/quotas so maybe is another fine tunning I need to work. I really appreciate any help.

4

@angieo any luck with resolving this issue?