I’m Non-English speaker. I apologize for poor English.
Been using on-premise(using docker) w/o any problems for 5 months.
A week ago I noticed heavy cpu consumption increment, and super-bunch of logs like
- 06:13:00 [ERROR] sentry.errors.events: preprocess.failed.emptyy (cache_key=‘e:6d2bacde6f50494fae71d6bd4901f272:69’)
this happens.
Googling didn’t save me from those problem and I didn’t know what to do.
So I decide to scale-out sentry.
Running 1 web, redis, postgre, worker, cron on a server before.
Added a web server under load balancer, and each server has 3 workers now.
I also did redis tuning suggested on this page
-
https://docs.sentry.io/server/performance/
Noooothing Helps. Nothing changed.
Anyway my sentry was not that bad and not my main question. It’s just for explanation of what I did last week.
–
Real problem is :
Today, it seems all sentry data is gone in sentryweb. Only data of recent 2 hour remained.
I didn’t run sentry cleanup for sure.
I thought it as cache-related-what problem so flushed redis, didn’t work.
Restarted(docker-compose down - up ) whole sentry docker. Even recent data is gone again.
Restarting didn’t work like this before. I did it many times and never lost any data.
Now every time I restart sentry, I lose all data.
But DB seems fine.
I still have old data in postresql, and I can look through old events in issue page with ‘older’ button.
And my issues have large number of event happens
argh… maybe some images are better than my poor sentences.
desc1) I lose all data all of a sudden, and now everytime I restart sentry, i lose it all
desc2)I have an issue with that 360k times happen by 1 user… while it happens 400 per hour