HTTP Error 413: Request Entity Too Large

SDKs
1

Many of our critical error messages are being dropped with the following error:

2017-07-13 17:16:49,394   203 ERROR     682 base                | Sentry responded with an error: HTTP Error 413: Request Entity Too Large (url: https://sentry.io/api/<REDACTED ID>/store/)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/raven/transport/threaded.py", line 172, in send_sync
    super(ThreadedHTTPTransport, self).send(url, data, headers)
  File "/usr/local/lib/python2.7/dist-packages/raven/transport/http.py", line 43, in send
    ca_certs=self.ca_certs,
  File "/usr/local/lib/python2.7/dist-packages/raven/utils/http.py", line 66, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 413: Request Entity Too Large

We are using python + django + celery, raven version 6.1.0. This particular error happened inside a celery task.

Having a truncated error message is fine, but entirely dropping it defeats the whole point of using Sentry.

I am guessing that we need to limit the size of the message on the client side? How can we do that?

I don’t want to disable breadcrumbs, but would be okay with dropping earlier or larger breadcrumbs if too large, replacing them with a “dropped” message or something like that.

2

The size limit for a Sentry report is currently 100 KB.

How many breadcrumbs are you passing? And are you passing a lot of free form extra data?

3

I am not sure how many breadcrumbs are being passed for the errors that get dropped, but I suspect it’s over 100 breadcrumbs. As far as I know, we aren’t using “extra”, unless it is being used automatically by Django or Celery.

Is there a way to limit the number of breadcrumbs? I couldn’t find any parameters for this in the settings.

This is the part of our django+celery settings where we configure raven:

INSTALLED_APPS += ['raven.contrib.django.raven_compat']
RAVEN_MIDDLEWARE = [
    'raven.contrib.django.raven_compat.middleware.Sentry404CatchMiddleware',
    'raven.contrib.django.raven_compat.middleware.SentryResponseErrorIdMiddleware',
]
MIDDLEWARE_CLASSES = RAVEN_MIDDLEWARE + MIDDLEWARE_CLASSES

SENTRY_CLIENT = 'raven.contrib.django.raven_compat.DjangoClient'
SENTRY_DSN = env('DJANGO_SENTRY_DSN')
SENTRY_CELERY_LOGLEVEL = logging.WARNING

RAVEN_CONFIG = {
    'dsn': SENTRY_DSN,
    'release': RELEASE_VERSION,
    'environment': 'prod',
    'ignore_exceptions': ['exceptions.KeyboardInterrupt'],
    'processors': ('raven.processors.SanitizePasswordsProcessor',),
    'CELERY_LOGLEVEL': SENTRY_CELERY_LOGLEVEL,
}
if CONTAINER_NAME:
    RAVEN_CONFIG['site'] = CONTAINER_NAME

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'root': {
        'level': 'INFO',
        'handlers': ['sentry', 'console'],
    },
    'formatters': {
        'verbose': {
            'format': '%(asctime)-15s %(process)5d %(levelname)-8s '
                      '%(lineno)4d %(module)-20s| %(message)s'
        },
    },
    'handlers': {
        'sentry': {
            'level': 'ERROR',
            'class': 'raven.contrib.django.raven_compat.handlers.SentryHandler',
        },
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'verbose'
        }
    },
    'loggers': {
        'django.db.backends': {
            'level': 'ERROR',
            'handlers': ['console'],
            'propagate': False,
        },
        'raven': {
            'level': 'DEBUG',
            'handlers': ['console'],
            'propagate': False,
        },
        'sentry.errors': {
            'level': 'DEBUG',
            'handlers': ['console'],
            'propagate': False,
        },
        'django.security.DisallowedHost': {
            'level': 'ERROR',
            'handlers': ['console', 'sentry'],
            'propagate': False,
        },
        'backoff': {
            'level': 'ERROR',
            'handlers': ['console'],
            'propagate': False
        }
    },
}
4

SDKs should have a “max breadcrumbs” configuration. If Python doesn’t have it, we’ll add it.