Python SDK extra data capped at < 400 characters

SDKs
#1

Hi! I’m upgrading from Raven to the Sentry Python SDK, and I’m noticing that my extra context consistently seems to be trimmed quite substantially. The true value is about 7546 characters long, and the value showing up in the Sentry report is a trimmed version amounting to about 359 characters. This particular item is actually a dict (so it appears in the JSON file as an object in its own right), so to be more precise, the Item I passed to scope.set_extra() is a dict of 138 items, which then comes through in the report as just 11 items.

From this topic as well as the documentation, I gather extra data gets cut off at 256k characters, or about 16k characters per item, and there’s an event payload maximum of 200 kB. But 7546 is well short of 16k or 200k, and the JSON file for this event’s currently around 23 kB.

Is there some other limitation with extra data I’m missing? This item contains information about user preferences that is extremely helpful debugging information.

#2

If you get the undesirable trimming behavior only when upgrading to the new SDK, the server-side limits are not the issue. Right now we trim each dictionary and array in extra to a maximum of ~10 items and to a depth of 5. SDK-side trimming is done to avoid building too large payloads that might slow down the entire application during JSON serialization, or create request bodies that may be above the 200 kB limit.

The limits relevant to your situation are declared here: https://github.com/getsentry/sentry-python/blob/8d669f6eba2a99410e656bd90efa064cf88259bb/sentry_sdk/serializer.py#L29-L30 There has been talk about adding options to configure those, but for now I would suggest monkeypatching them by importing the module and overwriting the constant.

1 Like
#3

I tried monkeypatching MAX_DATABAG_BREADTH and it worked, thanks so much for pointing me to that!

1 Like
#4

Is there any change on this? I’d like to increase the limit as well since we store stack-traces that get cut-off too eagerly now.

#5

For what it’s worth, in my case I’ve since noticed that the API has changed to deprecate set_extra in favor of set_context, and that when I use set_context, the data doesn’t seem to be capped as I originally described, at least not at a level that trims my data: TabbycatDebate/tabbycat#d7baa97, TabbycatDebate/tabbycat#16284b9

I couldn’t offer any insight into what’s changed or how this works, of course, but hope that helps!