You can create just a new DSN and delete old ones. You can find this in your project settings -> Client Keys.
Recommending storing the DSN somewhere private would be conflicting messaging from our side.
Let’s imagine you use @sentry/browser on your website, it’s impossible to hide the DSN in javascript. No one stops you from using the same DSN on your server.
So you see, we can’t make it secret so we don’t tell people it’s a secret, it’s basically an id identifying your account.