SSL Error when reload page

aamarques · August 20, 2021, 10:32am

Hey !
Help me to understand (and fix) why I’m getting those errors in web container ONLY when I reloar the Project page on Sentry UI.

web_1                                       | 2021-08-19T14:51:59.585404854Z Traceback (most recent call last):
web_1                                       | 2021-08-19T14:51:59.585543205Z   File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
web_1                                       | 2021-08-19T14:51:59.585557865Z     response = get_response(request)
web_1                                       | 2021-08-19T14:51:59.585565062Z   File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 126, in _get_response
web_1                                       | 2021-08-19T14:51:59.585571500Z     response = self.process_exception_by_middleware(e, request)
web_1                                       | 2021-08-19T14:51:59.585577798Z   File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 124, in _get_response
web_1                                       | 2021-08-19T14:51:59.585584096Z     response = wrapped_callback(request, *callback_args, **callback_kwargs)
web_1                                       | 2021-08-19T14:51:59.585590085Z   File "/usr/local/lib/python3.6/site-packages/sentry/../sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
web_1                                       | 2021-08-19T14:51:59.585596528Z     return callback(request, *args, **kwargs)
web_1                                       | 2021-08-19T14:51:59.585602346Z   File "/usr/local/lib/python3.6/site-packages/django/views/generic/base.py", line 68, in view
web_1                                       | 2021-08-19T14:51:59.585608371Z     return self.dispatch(request, *args, **kwargs)
web_1                                       | 2021-08-19T14:51:59.585634071Z   File "/usr/local/lib/python3.6/site-packages/django/views/generic/base.py", line 88, in dispatch
web_1                                       | 2021-08-19T14:51:59.585640279Z     return handler(request, *args, **kwargs)
web_1                                       | 2021-08-19T14:51:59.585645602Z   File "/usr/local/lib/python3.6/site-packages/sentry/web/frontend/base.py", line 556, in get
web_1                                       | 2021-08-19T14:51:59.585651123Z     photo_file = photo.getfile()
web_1                                       | 2021-08-19T14:51:59.585656158Z   File "/usr/local/lib/python3.6/site-packages/sentry/models/file.py", line 348, in getfile
web_1                                       | 2021-08-19T14:51:59.585661775Z     impl = self._get_chunked_blob(mode, prefetch)
web_1                                       | 2021-08-19T14:51:59.585667145Z   File "/usr/local/lib/python3.6/site-packages/sentry/models/file.py", line 340, in _get_chunked_blob
web_1                                       | 2021-08-19T14:51:59.585672473Z     delete=delete,
web_1                                       | 2021-08-19T14:51:59.585676087Z   File "/usr/local/lib/python3.6/site-packages/sentry/models/file.py", line 498, in __init__
web_1                                       | 2021-08-19T14:51:59.585679903Z     self.open()
web_1                                       | 2021-08-19T14:51:59.585683298Z   File "/usr/local/lib/python3.6/site-packages/sentry/models/file.py", line 535, in open
web_1                                       | 2021-08-19T14:51:59.585687066Z     self.seek(0)
web_1                                       | 2021-08-19T14:51:59.585690581Z   File "/usr/local/lib/python3.6/site-packages/sentry/models/file.py", line 599, in seek
web_1                                       | 2021-08-19T14:51:59.585700229Z     return self._seek(pos)
web_1                                       | 2021-08-19T14:51:59.585704426Z   File "/usr/local/lib/python3.6/site-packages/sentry/models/file.py", line 591, in _seek
web_1                                       | 2021-08-19T14:51:59.585708363Z     self._nextidx()
web_1                                       | 2021-08-19T14:51:59.585714666Z   File "/usr/local/lib/python3.6/site-packages/sentry/models/file.py", line 521, in _nextidx
web_1                                       | 2021-08-19T14:51:59.585718498Z     self._curfile = self._curidx.blob.getfile()
web_1                                       | 2021-08-19T14:51:59.585724136Z   File "/usr/local/lib/python3.6/site-packages/sentry/models/file.py", line 309, in getfile
web_1                                       | 2021-08-19T14:51:59.585727945Z     return storage.open(self.path)
web_1                                       | 2021-08-19T14:51:59.585731391Z   File "/usr/local/lib/python3.6/site-packages/django/core/files/storage.py", line 33, in open
web_1                                       | 2021-08-19T14:51:59.585735134Z     return self._open(name, mode)
web_1                                       | 2021-08-19T14:51:59.585738594Z   File "/usr/local/lib/python3.6/site-packages/django/core/files/storage.py", line 218, in _open
web_1                                       | 2021-08-19T14:51:59.585742336Z     return File(open(self.path(name), mode))
web_1                                       | 2021-08-19T14:51:59.585745793Z FileNotFoundError: [Errno 2] No such file or directory: '/data/files/37/40ed/db517b47468f38c8e2a4c1de7c'
web_1                                       | 2021-08-19T14:51:59.585749657Z 14:51:59 [ERROR] django.request: Internal Server Error: /avatar/2458d062793d4b1a8e6028246f139731/ (status_code=500 request=<WSGIRequest: GET '/avatar/2458d062793d4b1a8e6028246f139731/?s=120'>)
web_1                                       | 2021-08-19T14:51:59.629798480Z Traceback (most recent call last):
web_1                                       | 2021-08-19T14:51:59.629845363Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/connectionpool.py", line 600, in urlopen
web_1                                       | 2021-08-19T14:51:59.629853923Z     chunked=chunked)
web_1                                       | 2021-08-19T14:51:59.629859190Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/connectionpool.py", line 343, in _make_request
web_1                                       | 2021-08-19T14:51:59.629884330Z     self._validate_conn(conn)
web_1                                       | 2021-08-19T14:51:59.629890133Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/connectionpool.py", line 839, in _validate_conn
web_1                                       | 2021-08-19T14:51:59.629895418Z     conn.connect()
web_1                                       | 2021-08-19T14:51:59.629900795Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/connection.py", line 344, in connect
web_1                                       | 2021-08-19T14:51:59.629906216Z     ssl_context=context)
web_1                                       | 2021-08-19T14:51:59.629911327Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/util/ssl_.py", line 347, in ssl_wrap_socket
web_1                                       | 2021-08-19T14:51:59.629917031Z     return context.wrap_socket(sock, server_hostname=server_hostname)
web_1                                       | 2021-08-19T14:51:59.629923143Z   File "/usr/local/lib/python3.6/ssl.py", line 407, in wrap_socket
web_1                                       | 2021-08-19T14:51:59.629931318Z     _context=self, _session=session)
web_1                                       | 2021-08-19T14:51:59.629936637Z   File "/usr/local/lib/python3.6/ssl.py", line 817, in __init__
web_1                                       | 2021-08-19T14:51:59.629941759Z     self.do_handshake()
web_1                                       | 2021-08-19T14:51:59.629946822Z   File "/usr/local/lib/python3.6/ssl.py", line 1077, in do_handshake
web_1                                       | 2021-08-19T14:51:59.629952033Z     self._sslobj.do_handshake()
web_1                                       | 2021-08-19T14:51:59.629957264Z   File "/usr/local/lib/python3.6/ssl.py", line 689, in do_handshake
web_1                                       | 2021-08-19T14:51:59.629962961Z     self._sslobj.do_handshake()
web_1                                       | 2021-08-19T14:51:59.629968280Z ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)
web_1                                       | 2021-08-19T14:51:59.629974464Z
web_1                                       | 2021-08-19T14:51:59.629979556Z During handling of the above exception, another exception occurred:
web_1                                       | 2021-08-19T14:51:59.629984761Z
web_1                                       | 2021-08-19T14:51:59.629989920Z Traceback (most recent call last):
web_1                                       | 2021-08-19T14:51:59.629994882Z   File "/usr/local/lib/python3.6/site-packages/sentry/../sentry_sdk/transport.py", line 326, in send_event_wrapper
web_1                                       | 2021-08-19T14:51:59.630002642Z     self._send_event(event)
web_1                                       | 2021-08-19T14:51:59.630007897Z   File "/usr/local/lib/python3.6/site-packages/sentry/../sentry_sdk/transport.py", line 234, in _send_event
web_1                                       | 2021-08-19T14:51:59.630013306Z     headers={"Content-Type": "application/json", "Content-Encoding": "gzip"},
web_1                                       | 2021-08-19T14:51:59.630018853Z   File "/usr/local/lib/python3.6/site-packages/sentry/utils/sdk.py", line 216, in patched_send_request
web_1                                       | 2021-08-19T14:51:59.630024708Z     return _send_request(*args, **kwargs)
web_1                                       | 2021-08-19T14:51:59.630030374Z   File "/usr/local/lib/python3.6/site-packages/sentry/../sentry_sdk/transport.py", line 180, in _send_request
web_1                                       | 2021-08-19T14:51:59.630036429Z     headers=headers,
web_1                                       | 2021-08-19T14:51:59.630041873Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/request.py", line 72, in request
web_1                                       | 2021-08-19T14:51:59.630047361Z     **urlopen_kw)
web_1                                       | 2021-08-19T14:51:59.630054928Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/request.py", line 150, in request_encode_body
web_1                                       | 2021-08-19T14:51:59.630130816Z     return self.urlopen(method, url, **extra_kw)
web_1                                       | 2021-08-19T14:51:59.630146472Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/poolmanager.py", line 324, in urlopen
web_1                                       | 2021-08-19T14:51:59.630153731Z     response = conn.urlopen(method, u.request_uri, **kw)
web_1                                       | 2021-08-19T14:51:59.630159945Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/connectionpool.py", line 667, in urlopen
web_1                                       | 2021-08-19T14:51:59.630165721Z     **response_kw)
web_1                                       | 2021-08-19T14:51:59.630171128Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/connectionpool.py", line 667, in urlopen
web_1                                       | 2021-08-19T14:51:59.630177469Z     **response_kw)
web_1                                       | 2021-08-19T14:51:59.630182925Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/connectionpool.py", line 667, in urlopen
web_1                                       | 2021-08-19T14:51:59.630189097Z     **response_kw)
web_1                                       | 2021-08-19T14:51:59.630194496Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/connectionpool.py", line 638, in urlopen
web_1                                       | 2021-08-19T14:51:59.630200118Z     _stacktrace=sys.exc_info()[2])
web_1                                       | 2021-08-19T14:51:59.630205449Z   File "/usr/local/lib/python3.6/site-packages/sentry/../urllib3/util/retry.py", line 399, in increment
web_1                                       | 2021-08-19T14:51:59.630210993Z     raise MaxRetryError(_pool, url, error or ResponseError(cause))
web_1                                       | 2021-08-19T14:51:59.630216560Z urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='sentry.mycomp.org', port=443): Max retries exceeded with url: /api/1/store/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),))
web_1                                       | 2021-08-19T14:51:59.630222944Z 14:51:59 [ERROR] sentry_sdk.errors: Internal error in sentry_sdk

This host (modified of course) host='sentry.mycomp.org', port=443) has a valid certificate registered.

I’d appreciate any help and explanation.

Thanks

BYK · August 20, 2021, 12:12pm

Looks like you have an issue with missing avatar images and then Sentry is trying to log that error to itself over HTTPS and fails as you are using a custom CA and Sentry docker image (not your host system) does not know about or trust that CA.

First of all, I’m quite surprised that Sentry is trying to communicate with itself over the system.url-prefix. Could you have modified the system.internal-url-prefix setting in your config.yml?

Secondly, you can use the newly introduced custom CA support to alleviate the SSL issue for now: https://develop.sentry.dev/self-hosted/custom-ca-roots/

aamarques · August 20, 2021, 1:14pm

here the part of config.yml

system.internal-url-prefix: 'http://web:9000'
symbolicator.enabled: true
symbolicator.options:
  url: "http://symbolicator:3021"

but DST Root CA X3 is part of bundled, no ?
lrwxrwxrwx 1 root root 53 Jun 23 15:21 /etc/ssl/certs/DST_Root_CA_X3.pem → /usr/share/ca-certificates/mozilla/DST_Root_CA_X3.crt

Why I need to configure Custom root CA ?

Thks for your time @BYK

aamarques · August 26, 2021, 9:29am

Any ideas @BYK ?

BYK · August 26, 2021, 3:32pm

Ah, sorry my bad. I got confused as you shared your local system’s CA certs and I did not recognize this was a standard CA.

I do not know why Sentry cannot talk to your system.url-prefix but can you try setting the following in your sentry.conf.py file:

SENTRY_ENDPOINT='http://nginx'

aamarques · September 21, 2021, 10:04am

Hey @BYK sorry for the time I take to answer this.
The problem is that our system.url-prefix is an URL accessed only from external.
Changing the SENTRY_ENDPOINT='http://nginx' in sentry.conf.py solve the problem.

Thanks so much again!

system · October 6, 2021, 10:04am

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Internal error in sentry_sdk (Tunnel connection failed: 503 Service Unavailable) On-Premise	5	3348	December 1, 2020
Internal error in sentry_sdk On-Premise	13	5064	December 13, 2021
WrongBackend exception when trying to connect GitHub On-Premise	1	726	January 13, 2019
Password authentication failed for user "postgres" On-Premise	2	4213	February 15, 2017
Upgrading to the latest version from 9.1.2 On-Premise	13	2306	October 12, 2020

SSL Error when reload page

Related topics