Source code not found on JS files

[jan.dolezal]

Hi,

we run on-premise Sentry and we’re facing issues with incorrect mapping *.map files to original source files .ts/.js.

Every release i create new version, upload source maps with sentry-cli binary to this version, but if error will occure, we got error “Source code was not found” and Sentry wants to find them on website, instead on release artifacts.

Source maps are working fine, if they’re on the website. When we delete them, Sentry won’t find them. I tried even disable “Enable JavaScript source fetching” on project, but it didn’t helped.

Source maps are working fine even in development, so i think there is not problem in source maps.

We even upload dist JS files (our source is *.ts). Do we need to upload them?

Server version: 9.1.2
Client SDK version: sentry.javascript.browser:6.0.0

I’ll attach images fe. for version 2021.124.3.

Inside file ng-app.js is correct sourcemap URL //# sourceMappingURL=ng-app_353a961baecf34d71635.js.map. This file is on this release as shown from picture.

We create version and upload source maps this way:

sentry-cli releases new -p "userweb-static" "2021.124.3"
sentry-cli releases finalize "2021.124.3"
sentry-cli releases files "2021.124.3" upload-sourcemaps dist/js
sentry-cli releases -p "userweb-static" set-commits --auto "2021.124.3"

Next image (i can upload only 1): https://i.ibb.co/6H3XPtz/1.jpg

Is there issue with our workflow or with Sentry?

Thanks
John

[BYK]

Pinging @kamilogorek and @untitaker for help.

[kamilogorek]

Change line 3 to:

sentry-cli releases files "2021.124.3" upload-sourcemaps dist/js --url-prefix "~/js"

This change alone should fix it. Notice that your frames are pointing to /js/ng-app.js but your artifacts are uploaded as ~/ng-app.js (~/ is a protocol globbing matcher here, so it’ll match the initial / part of the URL). They both need to match to resolve correctly.

[jan.dolezal]

Thank you @kamilogorek .

But unfortunately, it didn’t help.

[kamilogorek]

Can you link an actual event please?

[jan.dolezal]

What do you mean “link”? Picture from event was attached and our Sentry is local only.

[kamilogorek]

Ah, right. My bad. I see that you are using environments in the SDK, can you try to specify --dist production in the upload-sourcemaps command?

[jan.dolezal]

Tried (environment/distribution is there), but sadly, no change

[kamilogorek]

At this point (assuming you triggered an event after artifacts upload), the only thing that can be wrong here is misconfigured on-premise instance, as everything else is setup correctly. cc @BYK

[jan.dolezal]

Well, we migrated to the cloud Sentry with Team plan and issue didn’t dissappeared

Link: https://sentry.io/organizations/seznam-cz/issues/2350067705/?project=5725840&query=is%3Aunresolved

[kamilogorek]

The release that this event is pointing to contains only .js.map files, but there is no .js file that stack trace frame is pointing to. There has to be a .js file that matches, and it has to contain a valid sourceMappingURL suffix which will be used for mapping.

[jan.dolezal]

Ok, i released new version with JS and MAP files, but no. Same issue.

Issue: https://sentry.io/organizations/seznam-cz/issues/2351861112/?project=5725840&query=is%3Aunresolved

JS sourceMappingURL is: //# sourceMappingURL=ng-app_f04760cd929fd8856de7.js.map.

Is it valid? When sourceMappingURL in js file has ~/js/ng-app_f04760cd929fd8856de7.js.map but artifact path is ng-app_f04760cd929fd8856de7.js.map.

This file (and sources also) is in release artifacts.

Or, isn’t problem that error originated from file with url query? js/ng-common.js?version=2021.142.0

[kamilogorek]

That’s really strange, I wonder if we might have a bug with query string handling.
One last thing that I’d like to ask you before digging into the Sentry source maps code itself. Can you specify dist: 'production' in the Sentry.init call as well? I can see environment is set, but dist is not, yet artifacts are uploaded to that distribution.

[jan.dolezal]

By the way i tried to remove query (trough RewriteFrames) and it didn’t helped.

You can see here https://sentry.io/organizations/seznam-cz/issues/2352239235/?project=5725840

There is now one more weird bug.

i’ll try that dist.

FYI, this is our init

import * as Sentry from '@sentry/browser';
import * as Integrations from '@sentry/integrations';

function sanitizeFileName(fileName: string) {
	return (
		fileName
			// uw-static statics requests
			.replace(/(.*)_[a-z0-9]{15,}\.js$/, '$1.js')
			// legacy userweb statics request
			.replace(/(.*)\?time.*/, '$1')
			.replace(/(.*)\?version.*/, '$1')
	);
}

if (process.env.NODE_ENV === 'production') {
	Sentry.init({
		dsn:
			'https://hidden',
		ignoreErrors: [
			// Random plugins/extensions
			'safari-extension',
			'__firefox__',
			'top.GLOBALS',
			// See: http://blog.errorception.com/2012/03/tale-of-unfindable-js-error.html
			'originalCreateNotification',
			'canvas.contentDocument',
			'MyApp_RemoveAllHighlights',
			'http://tt.epicplay.com',
			"Can't find variable: ZiteReader",
			'jigsaw is not defined',
			'ComboSearch is not defined',
			'http://loading.retry.widdit.com/',
			'atomicFindClose',
			// Facebook borked
			'fb_xd_fragment',
			// ISP "optimizing" proxy - `Cache-Control: no-transform` seems to reduce this. (thanks @acdha)
			// See http://stackoverflow.com/questions/4113268/how-to-stop-javascript-injection-from-vodafone-proxy
			'bmi_SafeAddOnload',
			'EBCallBackMessageReceived',
			// See http://toolbar.conduit.com/Developer/HtmlAndGadget/Methods/JSInjection.aspx
			'conduitPage',
		],
		// Best practices from https://gist.github.com/impressiver/5092952
		denyUrls: [
			// Facebook flakiness
			/graph\.facebook\.com/i,
			// Facebook blocked
			/connect\.facebook\.net\/en_US\/all\.js/i,
			// Woopra flakiness
			/eatdifferent\.com\.woopra-ns\.com/i,
			/static\.woopra\.com\/js\/woopra\.js/i,
			// Chrome extensions
			/extensions\//i,
			/^chrome:\/\//i,
			// Other plugins
			/127\.0\.0\.1:4001\/isrunning/i, // Cacaoweb
			/webappstoolbarba\.texthelp\.com\//i,
			/metrics\.itunes\.apple\.com\.edgesuite\.net\//i,
			/js_all_user/i,
			/moz-extension/i,
			/bootstrap\/js/i,
			/adform\.net/i,
		],
		maxBreadcrumbs: 20,
		release: `userweb-static@1.2.3`,
		integrations: [
			new Integrations.Angular(),
			new Integrations.RewriteFrames({
				iteratee(frame) {
					if (frame.filename) {
						/*
						 * Sentry is creating fingerprint depending on filename.
						 * If release has unique filename hash it will create new issue for already
						 * existing error.
						 */
						frame.filename = sanitizeFileName(frame.filename);
					}

					return frame;
				},
			}),
		],
	});

	addEventListener('unhandledrejection', (event) => {
		if (event.reason) {
			Sentry.captureException(event.reason);
		}
	});
}

[jan.dolezal]

Kamil, i tried to put dist: production and it works!

I don’t know if there were some changes on your side, but its resolved now.

Example: https://sentry.io/organizations/seznam-cz/issues/2373501783/?project=5725840&query=is%3Aunresolved

Thank you.

[rmilesson]

Hi!

We’re facing the same issue it seems like, however we cannot make it work by specifying the distribution unfortunately.

This is from a test project on our on premise Sentry.

The test error:
See the reply below this one since I can only put one media file/reply apparently.

The commands to create a release and upload sourcemaps:

sentry-cli releases new "$BUILD_VERSION"
sentry-cli releases files "$BUILD_VERSION" upload-sourcemaps build/static/js --url-prefix '~/static/js' --dist 'production'
sentry-cli releases finalize "$BUILD_VERSION"

This is the setup in the app (small test app generated with create-react-app):

import React from 'react';
import ReactDOM from 'react-dom';
import * as Sentry from '@sentry/react';
import { Integrations } from '@sentry/tracing';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';

Sentry.init({
  dsn: "[DSN_HERE]",
  integrations: [new Integrations.BrowserTracing()],
  environment: process.env.NODE_ENV,
  dist: 'production',

  // Set tracesSampleRate to 1.0 to capture 100%
  // of transactions for performance monitoring.
  // We recommend adjusting this value in production
  tracesSampleRate: 1.0,
});

ReactDOM.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>,
  document.getElementById('root')
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

I have no clue where it’s going wrong. I’ve been trying to see the logs from /src/sentry/lang/javascript/processor.py fetch_file since that would provide some clues at least but I have no idea where to find the logs.

Any ideas?

[rmilesson]

The error, referenced from the reply above this.

[kamilogorek]

@rmilesson you need to set a release in Sentry.init to the same value as $BUILD_VERSION to make it resolve.