Yeah, this can be a bit confusing at first. The public DSN is only really used for JavaScript, and it causes some different authentication behavior that wouldn’t apply to a Java desktop application.
Unless you’re a browser-based implementation you should use the full DSN.
The security implication is only that someone can send events on your behalf, which your only resolve is to revoke the API key and ship a new one. There’s no other access given by default with client keys.