Sentry.events.time-to-process timer maxes out at 3600<unit>

[kashyap.k]

Sentry version: 9.1.2

Okay, this one’s a weird one. During heavy event processing load, this metrics seem to max out at 3600ms (I’m assuming this logs ms). I tried purging the pending queue, after which the time to process increases as time goes by, which is baffling to me, up till this number.

No matter how many workers I keep increasing (or decreasing), this number remains. I’m thinking this might be some timeout that’s getting triggered, but not entirely sure.

I would’ve marked this down to some performance issue somewhere in the stack, but the graph shape is what prompted me to start this thread. In the duration of the flat graph, the event ingestion isn’t that form, and is more of a bell curve. I’d assume this graph would also resemble something similar unless there’s some strict cut off at 3600ms (or is it seconds?!)

[kashyap.k]

It may have been the writing down of the issue, but I realised this metric is seconds, not milliseconds. This makes much more sense now. I think once it reaches 1 hour the events expire, so that’s the maximum it’s going to reach.