Finetune event notifications based on event tag

1

Hey all,

my devs are generating way too many sentry alerts when working on the staging environment, which leads to my email inbox being overloaded with notifications.
I would like to switch off email notifs for staging sentry alerts but keep them for production environment.
The environment is flagged in one or two of the sentry events tags.
Any way I can use tags to fine-tune which email notifs I get?

Thanks in advance,
Arnaud

2

You can use the Rules tab within the Project Settings to add additional constraints to limit when notifications are sent, such a rule for when an event’s environment attribute equals production. These rules will be applied for notifications for all users, though — there’s no way to add any per-user filters. There are also some other caveats here: for example, the “first seen” condition applies to all environments — there’s no way send a notification when an event occurs for the first time in a specific environment.

We’re working on making “environment” a first-class concept on many parts of the application, including notifications, so more environment-based control over these notifications will likely be available in the future.

In the meantime, some users decide to split out their application’s production and staging environments into two separate projects, changing the DSN accordingly in their configuration files. This isn’t an ideal situation, but might work for you until there’s better environment support.

3

Of note we’re actively improving the Alerts functionality, both in terms of the user experience, as well as the capabilities of rules.

We have an initial step forward here:

https://github.com/getsentry/sentry/pull/3947

The goal is to take that, and continue to expand on the rule concepts to build out first class monitors, as well as build out per-env selection as @tkaemming mentioned.

4

Is there any further update regarding these features? In switching from Honeybadger to Sentry, our QA folks are concerned that they cannot now see, at a glance of their email, if a bug from staging was released to production.

EDITED: Let me further ask what the downsides are to making the different environments different projects?

1 Like
5

@tilthouse our goal is to make environments very powerful inside of a single project, but historically a lot of people have used separate projects for this behavior. Right now we’re focused on another piece of the core metadata, but environment is the priority beyond that.

6

In the future, would it be possible for the rules to be set on a per user filter basis as opposed to all users? That would be a great feature as we are using this environment feature now that it exists.

It would be nice to have rules set for different environments and notifications follow suit. i.e. less notifications for staging but more notifications for production environments respectively.

Thanks

7

Hi! Are there any updates on this issue?
We need to set notification rules for our project -
it could be great if we could also tune notifications for user groups:

  • for example so that the dev team could get notifications on testing/development environment, and the management team could get only pilot/prod environment issues. How we could possibly tune this right now? As it seems that the solution is to use different Sentry projects for this, which is not an ideal situation
1 Like
8

You’d have to setup Alert Rules, but you cant decide who gets what alerts. You could direct them to different e.g. Slack channels, but thats it right now.

9

hey @zeeg, I tried setting up Alert Rules, so that the Webhook notifications are only sent when it’s production environment.

So, what I did is that I set the following for staging:
image
and the following for production:

image
image617×524 22 KB

Basically, I was expecting that staging errors would only send mail notifications and production errors all notification types.

I also have my integration with “Alert Rule Action” enabled. So, what could I be missing here?

Btw, it would be great to have an option in Actions to “Dont send notifications”.

Has anyone tried this before?

PS: Here, my notification is going through a custom webhook notification. The other option I was looking for is to receive and analyze the environment tag in the issue.created payload. Unfortunately, I can see it’s not displayed there. Is there any way we could add that “easily”? We are using Sentry Cloud

Thank in advance,

10

This is my Alert rules configuration in the different environments. However, it’s sending the notification through the myintegration webhook regardless of everything:
image

Could this be a bug or something?

11

Did you try using only the environment selector field? I see that you have all environments + the explicit condition used. Try removing the condition and only using the env.

I also have my integration with “Alert Rule Action” enabled. So, what could I be missing here?

This is a newer integration not a legacy plugin, correct? If so I believe you need to specifically have it couple to the rules integration, and then it should show up as a selection in the action list. If its legacy then you can select webhooks from there (e.g. our built-in webhook plugin).

Btw, it would be great to have an option in Actions to “Dont send notifications”.

By default we don’t send any notifications unless you enable it via a rule. We have the default rule to send a notification for a new issues, but you could simply disable that.

12

Hi again @zeeg,

Pls check:
image
I left the configuration that says “event’s environment value equals production” and send notification to my new integration (not legacy plugin).

But it’s still sending the notification to my webhook even though the issue is set as staging environment.

image
image722×429 36.8 KB

What do you think? Am I missing something? or it’s a bug?

payload { action: ‘created’,
data:
{ issue:
{ platform: ‘node’,
lastSeen: ‘2020-02-26T16:09:09.079346Z’,
…
id: ‘1533717854’,
…
shortId: ‘NODEJS-T’,

13

Question, could it be related to the fact that the Alert condition is for Event and this is an Issue type?

If that’s the case, I didn’t see any other option to filter notifications by Environment for issues?

Update, I found an old post about this:

What it was interesting is that I can see an option over there to select the Environment. However, that option is not displayed anymore in my Sentry Cloud version. Was that part removed? Are you guys having plans to updated once again?

14

Hi epolancog!

Thanks for using Sentry - and for coming here for help.

I think what you’re experiencing here is a common issue among issue alerts. You’ve actually figured it out already:

Question, could it be related to the fact that the Alert condition is for Event and this is an Issue type?
The answer is yes. I think it may be a simpler mental model to think of these conditions as “OR” rather than “AND”.

So in the case of your last screenshot, without the “An issue is first seen” condition added, we’d expect you to get a notification for an issue if any event in that issue has environment=production. An issue could have 10 events, 5 of which have environment=production and environment=staging - and if an 11th came in, you’d get a notification under that rule.

However, in your second screenshot of the rules page, you have the condition “An issue is first seen” - and I would expect this to only notify you once. It seems like you’ve posted screenshots with 3 different states of rules and I’m not entirely sure on what alerts have notified you and when, so it’s hard to definitively draw any conclusions here. If you got an alert in this state, then I think that is something we’d need to investigate deeper.

And unfortunately, I believe you are correct that you cannot filter notifications by environment for “issues” (issue’s don’t have environment’s attached to them).

However, we are actually working on some new alerting features that will give you the power to do all of this and more. Stay tuned!

Please let me know if I’ve not cleared things up or if you have more questions. I’d be happy to continue helping to make sure you’re in a good spot :slight_smile: