Hey there, have been using self hosting since ~ 7.x you guys do a great job.
we are running a very minimal setup, so i understand the performance issues are purely due to the limitations we are imposing on our system on a t3 small. (2 VCPU 2gb ram).
but this is able to handle ~ 20 - 30 events per minute good enough ~ 30-50 projects. anyways i decided to turn on info logging for a particularly cron heavy project and left it running for a week or two.
Thats when our problems started with celery being unable to keep up with the demand, and 502/504s started getting thrown out left right and center. Granted i understand its these info logs and i turned them off to gain usability of our sentry system again (although i think we are still in a catch up phase in celery land ). We were not providing enough information for sentry to group these logs and EVERY SINGLE LOG was its own issue opps…
Now my question is do you happen to know if grouping these logs would have made any difference? or even if these logs were being grouped would we still have been in the same boat? its just we are at the point where we need to consider if we need to go from a ~40$ PM setup to a ~ 300$ setup PM which is a big jump
Also when everything was initially setup we made a seperate project for each environment i.e testing, approval, production. would doing it the correct way and providing the environment as part of the payload improve things much?
oh and its currently running 9.0.0 (incase there is known super bad bugs relating to any of these yes i really need to upgrade to 9.x.x )
incase someone stumbles across this, i was reviewing my bootstraper for the server and noticed that the redis service had issues since day one… and then because of this the “sentry cleanup” had not been running for the last 2 years . after 28 hours it was still deleting stale data. and i am now considering just starting a new at sentry 10 witha docker setup instead, depending on the outcome of this cron, if it ever finishes .
Alright the final follow up, not sure how useful this will be to anyone. But anywhoo, when this server was originally setup i created an ansible build script to bootstrap the server (so no docker whatsoever), this was half updated correctly for the transition from 7 -> 8 -> 9. (noticed issues with cron not running and stale data cloggin everything up), turns out there was also issues with systemctl AND supervisord spawning services, redis had issues persisting its DB. a whole lot of configuration issues opps…
After running the cron to completion, i was able to upgrade to 9.1.2 via pip2.7 install sentry=9.1.2 and migrate the DB with sentry upgrade (as empty as it now was ). i had issues doing this part due to the user context that initially installed all this… (spend way tooo long…)
At this point i had a working server again.
I tried going to sentry 10, but found out there are new dependencies i.e snubs and apon further inspection of the dockerfile snubs doesnt seem like a “out of the box and go” sort of thing, i also suspect there will be more than snubs. So i will be leaving this server at this point and moving across to a dockerised approach to take advantage of what is currently maintained by the sentry/opensouce team, and be upping the ram to the 3gb recommended minimum.
Even though there has been lack of replies i do appreciate all the work that is put into this project and the fact you guys still support onpremise as an option after all these years is a huuuge bonus
Hey @phyzical - BYK from the past/future here There was a time we were busy building out and did not have enough time to attend all the forum posts. Yours was one of them. I just wanted to say how much we appreciate folks sharing their own solutions or stories here, especially when all they here is crickets.
Sorry for the very late response and thanks a lot for sharing your experience with us and the community!
). We were not providing enough information for sentry to group these logs and EVERY SINGLE LOG was its own issue opps…
yes i really need to upgrade to 9.x.x )
. after 28 hours it was still deleting stale data. and i am now considering just starting a new at sentry 10 witha docker setup instead, depending on the outcome of this cron, if it ever finishes 