Update alert rules not work

I’ve edit and updated an alert rule, but it seems does not work, I can still get previous notifications even I’ve remove the notify action in the rule. I’ve tried to get some info from the source code but did not have any idea where to start. The deployed version is 22.2.0, and I’ll be very pleased to provide more infomation if it’s needed. BTW, if I want to debug this problem where should I start?