Problem with passing extra context from Celery

1

Hi!

I am trying to add some extra context to Sentry in my Celery task. I configured Celery according to https://docs.sentry.io/clients/python/integrations/celery/#setup (second version with celery.Celery base class), with one update – I stored raven client in self.raven_client instead of local var client.

Then I have custom Celery Task class, and inside run method I have something like this:

raven_client = self.app.raven_client

# several dummy values, just for test
raven_client.extra_context({'chleba': 'maslo'})
raven_client.user_context({'koza': 'roza'})
raven_client.http_context({'libor': 'milos'})
raven_client.tags_context({'jara': 'mara'})

I restart celery workers, run my task, receive message in Sentry, and found that just tags was captured (no extra info about user or in “Aditional data” section). So I begin with debugging. I trace whole path from my celery task up to Raven’s Client object. In all relevant places, context contain all of my extra informations (extra, user, …). Then message was send and when I checked Sentry interface again, I could see all of my extra informations

I was quite surprised, because I just add debuggers (from celery.contrib import rdb; rdb.set_trace()) on several places, nothing else.

So I remove debuggers, run my task once again, and Sentry shows me again just tags (no extra data in additional or user section):

sentry
sentry.png3000×1997 216 KB

There are several strange things:

  • tags are on right side only, or also on top of the page
  • there is some error box on top of page
  • there is different exception “stack” (last item is protected_call or trace_task)

I really don’t understand what is happening here. Do you have any explanation please?

2

I found something.

For some reason, two very similar exceptions occur in the single run of my task. Only first one contains rich context, second doesn’t have it. Due to natural aggregation of Sentry both events are displayed as a single one and only most recent event is rendered on page. If I click on “Older” button in event detail I could see the first event with my custom extra context.

3

Finally found the reason for this strange behaviour.

We have Django app using Sentry and Celery. According to Django integration docs we have 'raven.contrib.django.raven_compat', in INSTALLED_APPS and also extra Celery raven client (second variation based on Celery class).

And that is the problem. With this configuration we have two independent Raven clients watching for exceptions in app. In our case, if some exception occurs in Celery task, both clients process it and report as two individual events to Sentry service (and as I wrote above, only most recent is displayed on issue detail page, so it is quite easy to miss slightly older event with your context if you forget to click on “Older” button).

Because I wasn’t sure if I could safely remove one of Raven clients on our Celery worker nodes, I publish extra context not just on self.app.raven_client, but also on:

from raven.contrib.django.raven_compat.models import client as django_client
django_client.extra_context(extra_context)

This way, both events contain full context and I could finaly see it in Sentry issue page.