Merging a pull request doesn't do anything

I’ve linked my client’s GitHub repositories to Sentry. Creating a pull request and adding Fixes <id> adds a <person> marked this issue as fixed in ... comment, but merging it doesn’t seem to do anything. Shouldn’t merging a pull request mark an issue as resolved in the next release? What am I doing wrong?