Does using X-Sentry-Token request js map for each error?

If I upload my JS source map files and then only serve them if the X-Sentry-Token header is present, how does Sentry know when to fetch the map? If I fingerprint the JS file name, is Sentry smart enough to only fetch the map if it does not have one already with a matching fingerprint?