Decompressing gzipped json blob-as-text (nodestore_node table) data from on-premise instance of Sentry

we have an on-premise Sentry instance and we need to export all the event data. i’ve been able to export the tables that we are interested in and most are readable with the exception of the nodestore_node table. i think the data are stored as gzipped json blob-as-text, which i’m not familiar with. i’ve tried a bunch of different things but haven’t been successful in converting to something human readable. a few things i’ve tried so far:

  • SQL uncompress (just ended up with null for each row of converted_data)
    SELECT nodestore_node.data, CONVERT( UNCOMPRESS( nodestore_node.data ) USING 'utf8' ) AS converted_data FROM nodestore_node;

  • SQL cast (got error messages about CAST for this one)
    SELECT CAST( CAST( 'string' as XML ).value('.','varbinary(max)') AS varchar(max) )

  • decompress gzip in JS

is there a way that i could convert the blobs to something human readable? or is there an API to pull all the converted nodestore data?

any ideas would be greatly appreciated!! thank you!

Is this for back-up/restore or migration purposes? If so, instead of extracting event data, you should just be able to use the database files directly. Otherwise I think this api endpoint would help you more: https://docs.sentry.io/api/events/list-a-projects-events/

1 Like

that would work great. i tried using the DSN authentication but that didn’t work. what’s the auth endpoint to generate a token locally?

thank you!

Would this help: https://docs.sentry.io/api/auth/#auth-tokens

We also do something similar here in our onpremise tests, that said if you find that ugly and hard to decipher, I wouldn’t blame you :smiley:

1 Like

thank you!!

i think the auth token documentation (https://docs.sentry.io/api/auth/#auth-tokens) is for projects hosted by sentry io and not on-premise. but let me know if i’m missing something!

thanks for sharing the test - i ran the script line by line in my terminal, with updates for our environment set-up, but it just hangs on:

i also tried running the top part of the test script (again with environment updates) but got a CSRF error for the login.

any easy way you can think to generate a token locally?

thanks, again!

It applies to both as they use the same codebase. Just replace sentry.io parts with your own instance’s URL.

You don’t need the event generation parts. This was just to give you an idea about how to interact with the API, that said it is using cookie authentication so probably not very appropriate for your use case.

1 Like

ohhh - amazing!!! thank you so much !

1 Like