Sentry Spring Boot SDK

Hi there,
I’m trying to use Sentry in my Spring Boot + Gradle project and am going crazy.
I’m not seeing events and I expect that I should be.

I’m using:

compile 'io.sentry:sentry-spring-boot-starter:3.2.0'
compile 'io.sentry:sentry-logback:3.2.0'

I have an application.yml:

sentry:
  dsn: $secret-sentry-dsn
  in-app-includes: packages
  max-breadcrumbs: 150
  logging:
    minimum-event-level: info
    minimum-breadcrumb-level: debug
  enable-tracing: true
  traces-sample-rate: 1.0

and am using a profile called local-dev (application-local.yml):

sentry:
  environment: local-dev

I can see that when the application starts, traffic comes through:

But when I hit this endpoint:

@GetMapping("/crash")
    public String triggerException() {
        throw new RuntimeException(
                "Expected: controller used to showcase what " + "happens when an exception is thrown");
    }

I get no crashes through.

Can anyone tell me what I’m missing?

When using the old java SDK (1.7) I sent exceptions through a global exception handler, could this be interfering?

@ControllerAdvice
public class GlobalExceptionHandler {

    private final HoneycombContext honeycombContext;

    public GlobalExceptionHandler(HoneycombContext honeycombContext) {
        this.honeycombContext = honeycombContext;
    }

    @ResponseStatus(value = HttpStatus.BAD_REQUEST)
    @ExceptionHandler(value = BadRequestException.class)
    @ResponseBody
    ApiResponseMessage
    handleBadRequest(BadRequestException badRequestException) {
        Sentry.captureException(badRequestException);
        honeycombContext.getContextualData().put("errors.message", badRequestException.getMessage());
        return new ApiResponseMessage(HttpStatus.BAD_REQUEST.value(), badRequestException.getMessage());
    }

    @ResponseStatus(value = HttpStatus.NOT_FOUND)
    @ExceptionHandler(value = NotFoundException.class)
    @ResponseBody
    ApiResponseMessage
    handleResourceNotFoundException(NotFoundException notFoundException) {
        Sentry.captureException(notFoundException);
        honeycombContext.getContextualData().put("errors.message", notFoundException.getMessage());
        return new ApiResponseMessage(HttpStatus.NOT_FOUND.value(), notFoundException.getMessage());
    }

    @ResponseStatus(value = HttpStatus.FORBIDDEN)
    @ExceptionHandler(value = ForbiddenException.class)
    @ResponseBody
    ApiResponseMessage
    handleForbiddenException(ForbiddenException forbiddenException) {
        Sentry.captureException(forbiddenException);
        honeycombContext.getContextualData().put("errors.message", forbiddenException.getMessage());
        return new ApiResponseMessage(HttpStatus.NOT_FOUND.value(), forbiddenException.getMessage());
    }

    @ResponseStatus(value = HttpStatus.NOT_ACCEPTABLE)
    @ExceptionHandler(value = HttpMediaTypeNotAcceptableException.class)
    @ResponseBody
    ApiResponseMessage
    handleHttpMediaTypeNotAcceptableException(HttpMediaTypeNotAcceptableException httpMediaTypeNotAcceptableException) {
        Sentry.captureException(httpMediaTypeNotAcceptableException);
        honeycombContext.getContextualData().put("errors.message", httpMediaTypeNotAcceptableException.getMessage());
        return new ApiResponseMessage(HttpStatus.NOT_ACCEPTABLE.value(), httpMediaTypeNotAcceptableException.getMessage());
    }
}

I did not manage to reproduce your issue. Check the sample repo with same endpoint: https://github.com/maciej-scratches/sentry-unhandled-exception

Perhaps there is something specific to your project setup? Are you able to publish smallest sample that reproduces your issue?

I’ve looked at your config and my implementation matches that.

I’ve done some debugging, and using my global exception handlers, specifically, these method calls to the sentry SDK Sentry.captureException(badRequestException) lead to a chain where SentryClient.java's processEvent method returns a null event. Is that expected?

I wonder if there is a race condition in the error handling that comes out of the box, and what I’m doing in my global exception handlers?

It looks like the return value for processEvent is null because of a DuplicateHandler being triggered or present. I’m still getting familiar with the SDK code so not sure exactly how that works.

What do you think?

Edit:

I’ve since seen issue come through in sentry for packages I use. For example I also use Honeycomb in this project, and I can see that events triggered from Honeycomb come through. I can see Hikari thread starvation coming through as a result of me pausing the app for debugging.

Just not the exception data from code I’ve written.

All unhandled exceptions are captured by default in Sentry exception resolver for Spring so you do not need to capture them yourself in your controller advice. You can verify that, but I would bet that SentryClient#processEvent returns null because event gets deduplicated with DuplicateEventDetectionEventProcessor (maybe you can verify that with debug). Since this event gets deduplicated, the one triggered in exception resolver should be sent.

Turn please debug on with sentry.debug: true property and check if events are actually send to sentry. Perhaps they are filtered out in UI?

Thanks for that info, I’ll turn on debugging and see what I can find out.

So Sentry captures unhandled errors automatically now, has the behaviour around handled exceptions changed?

In my service classes I’ve got instances of throw new NotFoundException where NotFoundException is

public class NotFoundException extends RuntimeException {

    public NotFoundException() {
    }

    public NotFoundException(String message) {
        super(message);
    }

    public NotFoundException(String message, Throwable cause) {
        super(message, cause);
    }

    public NotFoundException(Throwable cause) {
        super(cause);
    }

    protected NotFoundException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) {
        super(message, cause, enableSuppression, writableStackTrace);
    }
}

These are then handled by my controller advice (GlobalExceptionHandler) where previously I had a call out to sentry which, in 1.7 looked like this Sentry.capture(NotFoundException); i.e

@ResponseStatus(value = HttpStatus.NOT_FOUND)
    @ExceptionHandler(value = NotFoundException.class)
    @ResponseBody
    ApiResponseMessage
    handleResourceNotFoundException(NotFoundException notFoundException) {
        **Sentry.capture(NotFoundException);**
        honeycombContext.getContextualData().put("errors.message", notFoundException.getMessage());
        return new ApiResponseMessage(HttpStatus.NOT_FOUND.value(), notFoundException.getMessage());
    }

Do I need to manually capture these still or does the new version do something else?

Here’s what I’ve gotten back with debug on:

DEBUG: Capturing event: 813b90c6e5ec42689d57b06956e1ff9f
INFO: Session is null on scope.withSession
2020-12-31 09:40:15.306  WARN 5663 --- [nio-8080-exec-5] .m.m.a.ExceptionHandlerExceptionResolver : Resolved [nz.ringfence.valuable.api.common.exceptions.NotFoundException: could not find company with id 2a689e52-f35b-4bda-901c-ea4f076bdc2c]
DEBUG: Capturing event: 3102dd2b70194605a3ccb49aacc47034
INFO: Session is null on scope.withSession
2020-12-31 09:40:15.599  INFO 5663 --- [/O dispatcher 2] n.r.v.a.c.c.h.LoggingResponseObserver    : Honeycomb upstream accepted: SimpleServerAccepted{rawHttpResponseBody=[91, 123, 34, 115, 116, 97, 116, 117, 115, 34, 58, 50, 48, 50, 125, 44, 123, 34, 115, 116, 97, 116, 117, 115, 34, 58, 50, 48, 50, 125, 44, 123, 34, 115, 116, 97, 116, 117, 115, 34, 58, 50, 48, 50, 125, 93, 10], eventMetadata={}, metrics=Metrics{clock=io.honeycomb.libhoney.transport.batch.impl.SystemClockProvider@95a76266, enqueueTime=842293597623, startOfHttpRequestTime=842397556299, endOfHttpRequestTime=842606481716}, batchData=SimpleBatchData{positionInBatch=0, batchStatusCode=200}, eventStatusCode=202, message='ACCEPTED'}
2020-12-31 09:40:15.600  INFO 5663 --- [/O dispatcher 2] n.r.v.a.c.c.h.LoggingResponseObserver    : Honeycomb upstream accepted: SimpleServerAccepted{rawHttpResponseBody=[91, 123, 34, 115, 116, 97, 116, 117, 115, 34, 58, 50, 48, 50, 125, 44, 123, 34, 115, 116, 97, 116, 117, 115, 34, 58, 50, 48, 50, 125, 44, 123, 34, 115, 116, 97, 116, 117, 115, 34, 58, 50, 48, 50, 125, 93, 10], eventMetadata={}, metrics=Metrics{clock=io.honeycomb.libhoney.transport.batch.impl.SystemClockProvider@95a76266, enqueueTime=842318707168, startOfHttpRequestTime=842397557337, endOfHttpRequestTime=842606482621}, batchData=SimpleBatchData{positionInBatch=1, batchStatusCode=200}, eventStatusCode=202, message='ACCEPTED'}
2020-12-31 09:40:15.601  INFO 5663 --- [/O dispatcher 2] n.r.v.a.c.c.h.LoggingResponseObserver    : Honeycomb upstream accepted: SimpleServerAccepted{rawHttpResponseBody=[91, 123, 34, 115, 116, 97, 116, 117, 115, 34, 58, 50, 48, 50, 125, 44, 123, 34, 115, 116, 97, 116, 117, 115, 34, 58, 50, 48, 50, 125, 44, 123, 34, 115, 116, 97, 116, 117, 115, 34, 58, 50, 48, 50, 125, 93, 10], eventMetadata={}, metrics=Metrics{clock=io.honeycomb.libhoney.transport.batch.impl.SystemClockProvider@95a76266, enqueueTime=842324079499, startOfHttpRequestTime=842397557898, endOfHttpRequestTime=842606483284}, batchData=SimpleBatchData{positionInBatch=2, batchStatusCode=200}, eventStatusCode=202, message='ACCEPTED'}
DEBUG: Envelope sent successfully.
DEBUG: Envelope flushed
DEBUG: Envelope sent successfully.
DEBUG: Envelope flushed

The sentry relevant entries appear to be:

DEBUG: Capturing event: 813b90c6e5ec42689d57b06956e1ff9f
INFO: Session is null on scope.withSession
DEBUG: Capturing event: 3102dd2b70194605a3ccb49aacc47034
INFO: Session is null on scope.withSession
DEBUG: Envelope sent successfully.
DEBUG: Envelope flushed
DEBUG: Envelope sent successfully.
DEBUG: Envelope flushed

Here’s my config as at the time of running:

application.yml

sentry:
  dsn: $DSN
  max-breadcrumbs: 150
  logging:
    minimum-event-level: warn
    minimum-breadcrumb-level: info
  enable-tracing: true
  traces-sample-rate: 1.0
  in-app-includes: nz.ringfence.valuable.api
  debug: true

and I’m using a profile called local, so application-local.yml contains:

sentry:
  environment: local-dev

Interestingly, here is a screenshot of the activity for the DSN:

The traffic there is from production, a few days ago. There is no traffic for the errors I have created as part of testing here.

Is it possible that the INFO: Session is null on scope.withSession is causing a problem that means no traffic is being set up? Do I need to manually create a session?

Created another sample project reflecting your code - events are coming in:

Is it possible that the INFO: Session is null on scope.withSession is causing a problem that means no traffic is being set up? Do I need to manually create a session?

No, there is no need to create sessions and sessions are not needed for traffic to come in.

Do I need to manually capture these still or does the new version do something else?

You don’t need to manually capture exceptions handled in controller advice. By default, all exceptions that have not been handled in the controller flow are sent to Sentry - even those that are resolved later on in the controller advice (this behaviour can be changed Spring Boot | Sentry Documentation).

So Sentry captures unhandled errors automatically now, has the behaviour around handled exceptions changed?

I believe this was the same in Sentry 1.x Spring integration, but if not - yes, Sentry handles unhandled exceptions automatically if Spring/Spring Boot integrations are set up.

I really appreciate the help Maciej. Thank you for taking the time!

In between working on this, I’ve been doing other bits and pieces in my project. As part of debugging other things, I saw some traffic come through.

I’m really scratching my head now as to what’s going on. It looks like Sentry is picking up the internals of the application, but nothing from my controllers.

I’m not really sure what to do at this point. My project matches the one you uploaded in your post, and yet I’m not getting any data through from my controllers - I am however getting other things through from the spring platform and the honeycomb dependency I’m also trialling.

Could it be that ones that don’t show up are filtered out in the UI? Pinging @bruno-garcia

Do you see events rate limited in the Stats page? Like:

There are different places that data can be dropped. The SDK could be dropping if the transport isn’t fast enough to flush them out. The Java transport has a queue of 30 items but not long ago was only 10.

Version 3.1.3 moved to 30 so you should be fine.

What’s the max throughput in requests per second do you have on this server? Could be that some bursts are caught up in this backpressure mechanism and end up dropped. You could try making this queue larger through setMaxQueueSize.

When the event makes into Sentry, it could be dropped due to spike protection which is there to help you save your quota. Or some other filters like inbound filters.

The SDK has a debug mode but if this is a high throughput server you’d need to add a customer logger to it and dump somewhere as obviously looking at the console wouldn’t help.

With this advice, I’ve fixed the problems!
Here’s my org stats page - I found that there were events being filtered out.

Looking at the inbound filter settings I saw that the toggle for filtering out events from localhost was toggled. I don’t think I’ve ever seen that setting before so not sure how it was toggled on, but that’s okay. Fixed now and have turned it off - I want stuff coming through during local host testing as it goes to it’s own environment anyway.

One question I now have is about the behaviour of these filters and of the SDK - all of the traffic coming through in my previous post was also during local testing, so how come the filter ate up the traffic from my controllers, but not traffic from spring? All of that occurred on my local machine. How did the internal errors of spring avoid being filtered along with the rest of it?

This is probably the thing that led me astray the most - I did think about UI filtering early on, but when I saw events coming through I thought that this could not be the case.

Also, FWIW I had to re-add Sentry.captureException(notFoundException); to my @ControllerAdvice i.e.

@ResponseStatus(value = HttpStatus.NOT_FOUND)
    @ExceptionHandler(value = NotFoundException.class)
    @ResponseBody
    ApiResponseMessage
    handleResourceNotFoundException(NotFoundException notFoundException) {
        Sentry.captureException(notFoundException);
        honeycombContext.getContextualData().put("errors.message", notFoundException.getMessage());
        return new ApiResponseMessage(HttpStatus.NOT_FOUND.value(), notFoundException.getMessage());
    }

in order for these events created by the controller to come through to the UI :slight_smile:

Thanks so much for all your help, Bruno and Maciej. We recently decided to implement Sentry in our software at my work, and I’m glad I pushed for that decision. You are all awesome and I love the product!

2 Likes

One question I now have is about the behaviour of these filters and of the SDK - all of the traffic coming through in my previous post was also during local testing, so how come the filter ate up the traffic from my controllers, but not traffic from spring? All of that occurred on my local machine. How did the internal errors of spring avoid being filtered along with the rest of it?

I am not sure and I don’t want to guess. You can look into details of each reported event in the UI to see exactly what JSON structure has been ingested.


Maybe you can see the difference between events reported in two different ways.

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.