Deploying on-premise Sentry 10 on AWS

We currently have Sentry 9.1.2 running on AWS as follows:

ALB -> ASG (run web)
ASG (run worker)
ASG (max instance 1 - run cron)
Elasticache Redis
Postgresql RDS

If we move to 10, there may be not much point keeping the nicely scalable web instances/workers since we’ll also need to run kafka, zookeeper, etc. I was pondering something like:

ALB -> ASG (max instance 1 - run docker-compose except postgres)
Postgresql RDS

Obviously, if the instance went down/had to be replaced we’d lose a bunch of stuff, but at least we wouldn’t lose our data. Or would we? Will Sentry be happy spinning up if it’s ‘lost’ kafka/zookeeper/redis … ? Would it help to keep the redis around as well?

You would lose data. Redis keeps many in-flight data such as pending jobs/tasks, kafka and zookeeper may also be holding in-flight event data. I also don’t see you mentioning Snuba and Clickhouse. All the event data is now stored in Clickhouse so you’d need to ensure its data is also safe.

Thanks! I should have clarified that we don’t care too much about losing in-flight data, we’d just like to not lose historical data. So if we had a resilient clickhouse and RDS, we could get away with something sketchy for the rest and not lose historical data? Would Sentry by ‘happy’ spinning up with an empty zookeeper/kafka/redis/snuba/whatever?

(sorry, I’m not familiar with Sentry 10’s architecture - I was just looking at the ‘whoah, there’s a lot of stuff in docker-compose’)

1 Like

Just to make sure you understand the ramifications: you may lose very recent event data that is not fully committed to the datastore if you lose data from Kafka. If that’s fine, yeah with a resilient Postgres and Clickhouse datastore, you should be good.

Yeah it would as these are assumed to be ephemeral. All data required for a smooth run is stored in Postgres.

/cc @matt in case I’m spreading lies here.