Import data from old sentry v9.x to v10

On-Premise
1

Hi,

I have just setup new sentry on-premise setup “10.1.0.dev0 f3f52bd”. Earlier I was having v9.x

Now the question is, I want to restore project, user settings from backup ( pg_dump of “sentry” database. Also, I do have physical i.e. full postgres directory backup as well)

Please guide me steps or just the tables needs to be restored in new setup if at all that is possible

Regards,
Sagar Sonawane

2

Hi @sagar.s,

There are others who tried this and documented their experience over at https://github.com/getsentry/onpremise/issues/387. We didn’t have time to publish an official guide yet for migrating to a Docker-based setup yet but in short, if you are able to migrate your Redis and Postgres data into Docker volumes, the ./install.sh script and the Sentry migrations would take care of the upgrade from 9.1.2 to 10 automatically.

3

Thanks for the reply @BYK. But, my situation is little different. I didn’t have old working sentry setup but only postgres dump. I don’t care about rest of the event data but only project configurations, so how can I import only that much configuration and start fresh with v10.

4

It’s still not too different. I’d clone the on-premise repo, and run something like the following:

docker-compose run --rm -d sentry -v /local/path/to/dump:/backups postgres pg_restore /backups/my_data.dump

After that, you should have your data into Postgres and be able to run ./install.sh that should take care of the rest.

5

Ohk. Let me try that, I will update you about the result.

1 Like
6

Hi @BYK,

I have cloned the repo using git clone https://github.com/getsentry/onpremise.git and then run the command like
docker-compose run --rm -v /home/ubuntu/sentry-backup-pgsql/:/backups postgres pg_restore -v -c -C --file=/backups

But, it didn’t do anything. I have checked the container ( temp. container ) for any activity but it was not doing anything…

7

Could it be this: https://stackoverflow.com/a/48746818/90297

It is weird that there’s no activity on the container.

8

Yeah, I couldn’t understand what was the issue. So I go ahead and did physical backup ( postgres data folder copy ) and restored it in volume manually and then executed install.sh.

Here is the install.log

https://pastebin.com/7kJZgQcR

It fails with upgrade: 1: exec: /etc/sentry/docker-entrypoint.sh: not found

9

@BYK I have decided to go for fresh install giving up old project configuration. I did prune all volumes, containers, network etc but still getting same above error
upgrade: 1: exec: /etc/sentry/docker-entrypoint.sh: not found.

I suspect a bug!!

10

@BYK I’m on latest pull and this issue seems to be solved in git repo c95eb56
but I’m getting same error and yes I’m running install script like SENTRY_IMAGE=getsentry/sentry:10 ./install.sh

11

I have gave up old data and executed install.sh without SENTRY_IMAGE parameter and setup proceeded

1 Like
12

Had the same issue during the migration.
But I managed it working in two steps:

  1. Staying in 9.x, upgrade Postgres to 9.6 with pgdump and back.
  2. Checkout master from the repo and then run ./install.sh

The most challenging to me was the fact, that new version is 1,5 times fatter:
2 threads of Kafka up to 1Gi each + Clickhouse 2Gi = 4Gi but not 2Gi as described in the Readme. In real load it is 6Gi average.

13

Thank you @Storager for sharing. But my older sentry installation was already blew up, so all I had was a postgres data directory.

Yeah, I second your observation. It is fat since stack has grown a lot, it’s quite a overhaul.

Well if we get smooth, stable and efficient sentry then it’s worth it.

14

Yes, sorry this is a known issue when trying to install 10.0.0 explicitly:

Hoping to get this fixed soon but in the meantime you can just drop the SENTRY_IMAGE override. (I now see that you did this but you could have preserved your old data)

The documented 2Gi is the minimum to get it running with minimal load. Of course the actual requirement will change based on your load.

15

No it was enough with v 9.0 try to run v10 on 2Gi without swap and check all te services. You will see a lot of issues on Internal project. Mostly from sentry api, but as also from clock house and Kafka.

16

Also there is no requirements about cpu mentioned. I would say 2 is a minimum to run this stack up somehow.

17

@Storager Yes, I have observed the same. I have posted about it in separate topic here https://forum.sentry.io/t/too-many-buffer-error-api-project-id-store/9525

18

Would you like to update these in a PR on the on-premise repo?